Jul 29 15:44:31.066528 ip-10-0-143-118 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 29 15:44:31.066542 ip-10-0-143-118 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 29 15:44:31.066551 ip-10-0-143-118 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 29 15:44:31.066882 ip-10-0-143-118 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 29 15:44:41.293189 ip-10-0-143-118 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 29 15:44:41.293209 ip-10-0-143-118 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 766b64a9bb9042d0b45db274afb44038 --
Jul 29 15:46:33.797762 ip-10-0-143-118 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 15:46:34.095938 ip-10-0-143-118 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 15:46:34.095938 ip-10-0-143-118 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 15:46:34.095938 ip-10-0-143-118 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 15:46:34.095938 ip-10-0-143-118 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 15:46:34.096558 ip-10-0-143-118 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 15:46:34.097629 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.097535    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 15:46:34.102344 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102328    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:46:34.102383 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102346    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:46:34.102383 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102351    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:46:34.102383 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102355    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:46:34.102383 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102359    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:46:34.102383 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102363    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:46:34.102383 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102368    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:46:34.102383 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102372    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:46:34.102383 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102377    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:46:34.102383 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102381    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:46:34.102383 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102385    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102389    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102393    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102397    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102402    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102406    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102409    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102414    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102419    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102425    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102429    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102433    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102437    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102441    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102445    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102449    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102452    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102455    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102459    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102462    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:46:34.102677 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102466    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102470    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102473    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102477    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102480    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102483    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102487    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102490    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102495    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102499    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102516    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102521    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102525    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102530    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102534    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102538    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102608    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102616    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102620    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102629    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:46:34.103582 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102633    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102637    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102644    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102650    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102654    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102658    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102662    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102672    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102677    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102681    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102685    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102692    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.102696    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.103975    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.103995    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104001    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104007    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104012    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104047    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:46:34.104195 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104055    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104155    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104163    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104167    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104171    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104176    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104181    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104185    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104189    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104194    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104198    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104203    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104207    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104211    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104215    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104219    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104224    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104227    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104231    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:46:34.104679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104234    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:46:34.105136 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104237    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:46:34.105136 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104241    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:46:34.105136 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104245    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:46:34.105136 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104249    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:46:34.105136 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104252    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:46:34.105136 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104256    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:46:34.105136 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104259    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:46:34.105136 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104263    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:46:34.105136 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104267    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:46:34.105136 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.104271    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105281    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105291    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105296    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105300    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105304    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105308    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105312    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105316    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105320    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105323    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105327    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105331    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105334    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105338    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105341    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105345    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105349    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105353    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105356    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:46:34.105374 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105361    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105364    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105368    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105372    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105375    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105379    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105386    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105390    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105395    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105399    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105404    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105408    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105411    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105415    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105419    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105422    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105426    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105429    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105433    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:46:34.105883 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105436    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105440    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105445    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105450    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105454    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105459    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105462    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105467    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105470    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105474    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105477    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105481    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105484    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105488    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105491    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105497    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105501    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105520    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105525    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105529    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:46:34.106349 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105533    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105536    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105540    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105543    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105547    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105553    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105556    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105560    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105563    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105566    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105570    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105574    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105578    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105581    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105585    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105588    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105594    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105599    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105604    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:46:34.106867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105607    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105611    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105614    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105619    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105622    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105626    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105630    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105633    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105637    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105641    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105644    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105648    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105653    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105657    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105661    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105666    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105669    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105673    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105677    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105682    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:46:34.107357 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105685    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.105689    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106235    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106242    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106247    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106251    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106256    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106260    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106264    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106269    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106273    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106277    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106281    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106285    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106290    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106294    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106298    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106303    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106309    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106313    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:46:34.107863 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106318    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106322    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106326    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106331    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106335    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106339    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106343    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106347    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106351    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106355    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106359    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106365    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106369    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106372    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106376    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106380    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106384    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106388    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106392    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106395    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:46:34.108354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106400    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106404    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106407    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106411    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106415    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106418    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106423    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106426    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106430    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106434    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106437    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106441    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106445    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106449    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106453    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106457    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106460    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106464    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106468    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106472    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:46:34.108905 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106476    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106480    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106483    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106487    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106491    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106495    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106499    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106517    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106521    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106525    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106530    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106534    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106538    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106542    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106546    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106550    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106554    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106558    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106562    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106566    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:46:34.109450 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106569    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106573    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106577    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106580    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106587    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106590    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106594    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106598    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106601    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106605    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106608    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106613    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106617    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106621    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106624    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106628    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106634    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106640    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106645    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:46:34.109946 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106650    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.106654    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106747    2560 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106757    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106765    2560 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106771    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106777    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106782    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106788    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106794    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106799    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106803    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106807    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106812    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106816    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106821    2560 flags.go:64] FLAG: --cgroup-root=""
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106827    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106833    2560 flags.go:64] FLAG: --client-ca-file=""
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106837    2560 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106842    2560 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106849    2560 flags.go:64] FLAG: --cluster-domain=""
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106853    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106858    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106863    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 15:46:34.110436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106867    2560 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106873    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106877    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106882    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106887    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106891    2560 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106895    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106901    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106906    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106910    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106916    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106920    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106924    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106929    2560 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106933    2560 flags.go:64] FLAG: --enable-server="true"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106937    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106944    2560 flags.go:64] FLAG: --event-burst="100"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106948    2560 flags.go:64] FLAG: --event-qps="50"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106953    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106957    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106961    2560 flags.go:64] FLAG: --eviction-hard=""
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106966    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106970    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106975    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106982    2560 flags.go:64] FLAG: --eviction-soft=""
Jul 29 15:46:34.111042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106988    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106992    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.106997    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107002    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107006    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107010    2560 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107014    2560 flags.go:64] FLAG: --feature-gates=""
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107019    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107024    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107028    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107032    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107036    2560 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107041    2560 flags.go:64] FLAG: --help="false"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107045    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107050    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107055    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107059    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107064    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107069    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107073    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107077    2560 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107081    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107085    2560 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 15:46:34.111662 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107090    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107094    2560 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107098    2560 flags.go:64] FLAG: --kube-reserved=""
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107102    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107107    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107111    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107115    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107119    2560 flags.go:64] FLAG: --lock-file=""
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107124    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107129    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107134    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107141    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107146    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107150    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107154    2560 flags.go:64] FLAG: --logging-format="text"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107158    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107163    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107167    2560 flags.go:64] FLAG: --manifest-url=""
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107171    2560 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107177    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107181    2560 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107187    2560 flags.go:64] FLAG: --max-pods="110"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107191    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107196    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107200    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 15:46:34.112255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107205    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107209    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107214    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107218    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107227    2560 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107231    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107236    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107241    2560 flags.go:64] FLAG: --pod-cidr=""
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107245    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107252    2560 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107257    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107261    2560 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107265    2560 flags.go:64] FLAG: --port="10250"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107269    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107274    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0abbeebc33e50154e"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107280    2560 flags.go:64] FLAG: --qos-reserved=""
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107284    2560 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107291    2560 flags.go:64] FLAG: --register-node="true"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107296    2560 flags.go:64] FLAG: --register-with-taints=""
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107300    2560 flags.go:64] FLAG: --registry-burst="10"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107305    2560 flags.go:64] FLAG: --registry-qps="5"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107309    2560 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107313    2560 flags.go:64] FLAG: --reserved-memory=""
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107318    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107322    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 15:46:34.112874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107327    2560 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107331    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107336    2560 flags.go:64] FLAG: --runonce="false"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107340    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107344    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107348    2560 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107352    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107357    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107362    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107367    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107371    2560 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107375    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107380    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107384    2560 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107389    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107393    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107397    2560 flags.go:64] FLAG: --system-cgroups=""
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107401    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107409    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107413    2560 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107417    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107423    2560 flags.go:64] FLAG: --tls-min-version=""
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107429    2560 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107433    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107438    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 15:46:34.113517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107442    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107446    2560 flags.go:64] FLAG: --v="2"
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107452    2560 flags.go:64] FLAG: --version="false"
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107457    2560 flags.go:64] FLAG: --vmodule=""
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107462    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.107467    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107648    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107655    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107660    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107664    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107668    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107671    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107675    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107679    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107684    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107687    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107692    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107695    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107698    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107702    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107706    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:46:34.114444 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107710    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107713    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107717    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107720    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107724    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107728    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107732    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107735    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107740    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107744    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107748    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107751    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107756    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107759    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107762    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107766    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107769    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107773    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107777    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107783    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:46:34.115354 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107787    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107792    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107796    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107799    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107804    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107808    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107811    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107815    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107819    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107823    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107826    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107830    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107833    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107836    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107840    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107843    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107847    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107850    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107854    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107857    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:46:34.116180 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107862    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107866    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107870    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107874    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107877    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107881    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107884    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107888    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107892    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107896    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107899    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107903    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107906    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107910    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107914    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107918    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107922    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107925    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107929    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:46:34.116728 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107932    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107938    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107944    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107948    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107952    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107955    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107959    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107963    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107968    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107973    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107978    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107982    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107987    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107992    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.107996    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.108000    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.108003    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.108007    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.108011    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:46:34.117579 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.108015    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.108018    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.108022    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.108026    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.108029    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.108032    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.108713    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.116031    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.116050    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.116658    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.116672    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116760    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116768    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116774    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116779    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:46:34.118305 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116784    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116789    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116793    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116798    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116803    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116808    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116812    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116817    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116821    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116825    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116830    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116835    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116839    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116845    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116851    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116856    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116861    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116865    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116870    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:46:34.118901 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116874    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116878    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116883    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116887    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116891    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116896    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116904    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116911    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116917    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116922    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116927    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116932    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116936    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116940    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116945    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116949    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116953    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116957    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116962    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:46:34.119686 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116966    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116970    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116974    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116979    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116984    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116988    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116993    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.116997    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117002    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117006    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117010    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117014    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117019    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117023    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117027    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117031    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117036    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117040    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117044    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117049    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:46:34.120163 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117055    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117059    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117063    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117068    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117072    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117077    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117081    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117086    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117090    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117094    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117099    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117103    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117107    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117112    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117116    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117120    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117124    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117129    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117133    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117137    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:46:34.120702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117141    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117146    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117150    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117154    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117159    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117163    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117167    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117172    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117175    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117193    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117198    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117203    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117208    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117213    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117217    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117221    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:46:34.121216 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117225    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.117233    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117412    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117419    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117423    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117428    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117432    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117436    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117440    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117444    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117449    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117453    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117457    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117461    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117465    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:46:34.121737 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117470    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117474    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117478    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117483    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117488    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117492    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117496    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117500    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117518    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117523    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117527    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117531    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117536    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117541    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117545    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117550    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117554    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117558    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117562    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:46:34.122234 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117567    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117571    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117575    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117579    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117583    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117587    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117592    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117597    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117602    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117606    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117610    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117614    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117618    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117623    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117627    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117631    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117636    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117640    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117647    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:46:34.122867 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117652    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117658    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117663    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117667    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117672    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117676    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117680    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117684    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117689    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117693    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117699    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117706    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117711    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117716    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117721    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117725    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117730    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117735    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117740    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117744    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:46:34.123367 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117749    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117753    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117757    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117762    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117766    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117770    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117774    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117778    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117782    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117786    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117791    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117795    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117798    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117803    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117807    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117811    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117815    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117822    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117826    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117831    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:46:34.123943 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117835    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:46:34.124490 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117840    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:46:34.124490 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117844    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:46:34.124490 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117849    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:46:34.124490 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117853    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:46:34.124490 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117857    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:46:34.124490 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117861    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:46:34.124490 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:34.117865    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:46:34.124490 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.117874    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 15:46:34.124490 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.118028    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 15:46:34.124490 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.122608    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 15:46:34.124490 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.123728    2560 server.go:1019] "Starting client certificate rotation"
Jul 29 15:46:34.124490 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.123837    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 15:46:34.124490 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.123877    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 15:46:34.125796 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.125780    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 15:46:34.127306 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.127287    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 15:46:34.136218 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.136201    2560 log.go:25] "Validated CRI v1 runtime API"
Jul 29 15:46:34.138164 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.138149    2560 log.go:25] "Validated CRI v1 image API"
Jul 29 15:46:34.138989 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.138976    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 15:46:34.143785 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.143762    2560 fs.go:135] Filesystem UUIDs: map[5c86b034-4bee-43c5-872b-c8a6fa144b44:/dev/nvme0n1p3 7926ec33-555a-4d33-8845-6f8437d582ea:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 29 15:46:34.143858 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.143781    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 15:46:34.148529 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.148415    2560 manager.go:216] Machine: {Timestamp:2026-07-29 15:46:34.147370433 +0000 UTC m=+0.255986291 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3090175 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d06105ab2edd6b55c738fa98d0a12 SystemUUID:ec2d0610-5ab2-edd6-b55c-738fa98d0a12 BootID:766b64a9-bb90-42d0-b45d-b274afb44038 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d7:eb:c8:4d:9b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d7:eb:c8:4d:9b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:4d:ca:00:db:e0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 15:46:34.148529 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.148522    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 15:46:34.148638 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.148628    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 15:46:34.151044 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.151015    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 15:46:34.151205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.151047    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-118.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 15:46:34.151700 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.151688    2560 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 15:46:34.151754 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.151704    2560 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 15:46:34.151754 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.151714    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 15:46:34.151754 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.151735    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 15:46:34.152494 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.152482    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 15:46:34.154341 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.154327    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 15:46:34.154473 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.154462    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 15:46:34.155008 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.154998    2560 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 15:46:34.155072 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.155014    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 15:46:34.155072 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.155034    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 15:46:34.155072 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.155047    2560 kubelet.go:398] "Adding apiserver pod source"
Jul 29 15:46:34.155072 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.155060    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 15:46:34.155444 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.155430    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 15:46:34.155529 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.155454    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 15:46:34.157566 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.157549    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 15:46:34.158360 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.158344    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 15:46:34.159258 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.159242    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 15:46:34.159344 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.159280    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 15:46:34.160738 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.160725    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 15:46:34.160806 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.160744    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 15:46:34.160806 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.160753    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 15:46:34.160806 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.160762    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 15:46:34.160806 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.160774    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 15:46:34.160806 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.160783    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 15:46:34.160806 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.160795    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 15:46:34.160806 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.160803    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 15:46:34.161032 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.160826    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 15:46:34.161032 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.160836    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 15:46:34.161032 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.160856    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 15:46:34.161032 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.160865    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 15:46:34.161439 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.161427    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 15:46:34.161488 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.161442    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 15:46:34.164529 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.164499    2560 server.go:1268] "Started kubelet"
Jul 29 15:46:34.167300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.167240    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 15:46:34.167584 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.167534    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 15:46:34.167656 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.167616    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 15:46:34.168069 ip-10-0-143-118 systemd[1]: Started Kubernetes Kubelet.
Jul 29 15:46:34.169438 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.169421    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 15:46:34.170964 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.170945    2560 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 15:46:34.172786 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.172767    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 15:46:34.172870 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.172792    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 15:46:34.172923 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.172892    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 15:46:34.173139 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.173119    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 15:46:34.173539 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.172696    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-118.ec2.internal.18c6cd43fb78b035  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-118.ec2.internal,UID:ip-10-0-143-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-118.ec2.internal,},FirstTimestamp:2026-07-29 15:46:34.164473909 +0000 UTC m=+0.273089748,LastTimestamp:2026-07-29 15:46:34.164473909 +0000 UTC m=+0.273089748,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-118.ec2.internal,}"
Jul 29 15:46:34.176365 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.176346    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 15:46:34.176365 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.176349    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 15:46:34.176532 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.176516    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-118.ec2.internal\" not found"
Jul 29 15:46:34.176870 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.176856    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 15:46:34.176975 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.176856    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 15:46:34.177045 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.177037    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 15:46:34.178273 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.177178    2560 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 15:46:34.178273 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.177191    2560 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 15:46:34.178655 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.178637    2560 factory.go:55] Registering systemd factory
Jul 29 15:46:34.178737 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.178681    2560 factory.go:223] Registration of the systemd container factory successfully
Jul 29 15:46:34.178999 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.178986    2560 factory.go:153] Registering CRI-O factory
Jul 29 15:46:34.178999 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.179004    2560 factory.go:223] Registration of the crio container factory successfully
Jul 29 15:46:34.179122 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.179066    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 15:46:34.179122 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.179090    2560 factory.go:103] Registering Raw factory
Jul 29 15:46:34.179122 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.179105    2560 manager.go:1196] Started watching for new ooms in manager
Jul 29 15:46:34.179490 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.179478    2560 manager.go:349] Starting recovery of all containers
Jul 29 15:46:34.185259 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.185230    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 15:46:34.185431 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.185406    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 15:46:34.191040 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.191030    2560 manager.go:354] Recovery completed
Jul 29 15:46:34.195142 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.195124    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5wdqc"
Jul 29 15:46:34.197222 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.197209    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:46:34.199188 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.199172    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:46:34.199250 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.199199    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:46:34.199250 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.199209    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:46:34.199675 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.199658    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 15:46:34.199675 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.199674    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 15:46:34.199768 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.199693    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 15:46:34.199768 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.199666    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:46:34.200495 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.200480    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:46:34.200577 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.200538    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:46:34.200577 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.200555    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:46:34.201171 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.201107    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-118.ec2.internal.18c6cd43fd8a5d81  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-118.ec2.internal,UID:ip-10-0-143-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-118.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-118.ec2.internal,},FirstTimestamp:2026-07-29 15:46:34.199186817 +0000 UTC m=+0.307802656,LastTimestamp:2026-07-29 15:46:34.199186817 +0000 UTC m=+0.307802656,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-118.ec2.internal,}"
Jul 29 15:46:34.201316 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.201304    2560 policy_none.go:49] "None policy: Start"
Jul 29 15:46:34.201356 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.201323    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 15:46:34.201356 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.201338    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 15:46:34.202348 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.202337    2560 policy_none.go:47] "Start"
Jul 29 15:46:34.202793 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.202781    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5wdqc"
Jul 29 15:46:34.242217 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.242060    2560 manager.go:337] "Starting Device Plugin manager"
Jul 29 15:46:34.242321 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.242241    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 15:46:34.242321 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.242255    2560 server.go:87] "Starting device plugin registration server"
Jul 29 15:46:34.242468 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.242455    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 15:46:34.242530 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.242469    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 15:46:34.242597 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.242581    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 15:46:34.242691 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.242678    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 15:46:34.242771 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.242693    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 15:46:34.243218 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.243192    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 15:46:34.243291 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.243247    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-118.ec2.internal\" not found"
Jul 29 15:46:34.342712 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.342691    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:46:34.343783 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.343615    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:46:34.343783 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.343640    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:46:34.343783 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.343651    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:46:34.343783 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.343678    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.352075 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.351999    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.352075 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.352027    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-118.ec2.internal\": node \"ip-10-0-143-118.ec2.internal\" not found"
Jul 29 15:46:34.367769 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.367745    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-118.ec2.internal\" not found"
Jul 29 15:46:34.368898 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.368871    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 15:46:34.370038 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.370017    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 15:46:34.370114 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.370050    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 15:46:34.370114 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.370068    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 15:46:34.370114 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.370096    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 15:46:34.373729 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.373712    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:46:34.467989 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.467958    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-118.ec2.internal\" not found"
Jul 29 15:46:34.471131 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.471113    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-118.ec2.internal"]
Jul 29 15:46:34.471196 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.471182    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:46:34.472687 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.472669    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:46:34.472788 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.472702    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:46:34.472788 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.472718    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:46:34.474183 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.474167    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:46:34.474329 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.474315    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.474381 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.474351    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:46:34.474877 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.474861    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:46:34.474947 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.474878    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:46:34.474947 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.474892    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:46:34.474947 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.474896    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:46:34.474947 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.474903    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:46:34.475064 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.474906    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:46:34.476056 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.476042    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.476106 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.476072    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:46:34.476718 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.476701    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:46:34.476811 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.476731    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:46:34.476811 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.476744    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:46:34.477949 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.477933    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/334f61d991df88637e87bb21bfbae838-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal\" (UID: \"334f61d991df88637e87bb21bfbae838\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.478021 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.477955    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/334f61d991df88637e87bb21bfbae838-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal\" (UID: \"334f61d991df88637e87bb21bfbae838\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.478021 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.477969    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cf62d983a3713d310961fc20664891a3-config\") pod \"kube-apiserver-proxy-ip-10-0-143-118.ec2.internal\" (UID: \"cf62d983a3713d310961fc20664891a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.502142 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.502122    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-118.ec2.internal\" not found" node="ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.506320 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.506305    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-118.ec2.internal\" not found" node="ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.568325 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:34.568302    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-118.ec2.internal\" not found"
Jul 29 15:46:34.578366 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.578346    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/334f61d991df88637e87bb21bfbae838-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal\" (UID: \"334f61d991df88637e87bb21bfbae838\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.578439 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.578372    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/334f61d991df88637e87bb21bfbae838-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal\" (UID: \"334f61d991df88637e87bb21bfbae838\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.578439 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.578391    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cf62d983a3713d310961fc20664891a3-config\") pod \"kube-apiserver-proxy-ip-10-0-143-118.ec2.internal\" (UID: \"cf62d983a3713d310961fc20664891a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.578439 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.578414    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/334f61d991df88637e87bb21bfbae838-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal\" (UID: \"334f61d991df88637e87bb21bfbae838\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.578562 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.578448    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/334f61d991df88637e87bb21bfbae838-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal\" (UID: \"334f61d991df88637e87bb21bfbae838\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:34.578562 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:34.578448    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cf62d983a3713d310961fc20664891a3-config\") pod \"kube-apiserver-proxy-ip-10-0-143-118.ec2.internal\" (UID: \"cf62d983a3713d310961fc20664891a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:36.335287 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:36.335209    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-118.ec2.internal\" not found"
Jul 29 15:46:36.435728 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:36.435696    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-118.ec2.internal\" not found"
Jul 29 15:46:36.471200 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:36.471163    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:36.475892 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:36.475874    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:36.536552 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:36.536521    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-118.ec2.internal\" not found"
Jul 29 15:46:36.637086 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:36.637016    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-118.ec2.internal\" not found"
Jul 29 15:46:36.674372 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:36.674350    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:46:36.737587 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:36.737561    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-118.ec2.internal\" not found"
Jul 29 15:46:36.790026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:36.789984    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 15:46:36.790484 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:36.790177    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 15:46:36.790484 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:36.790176    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 15:46:36.838497 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:36.838466    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-118.ec2.internal\" not found"
Jul 29 15:46:36.872648 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:36.872615    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 15:41:35 +0000 UTC" deadline="2028-04-12 03:03:38.75979075 +0000 UTC"
Jul 29 15:46:36.872648 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:36.872641    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14939h17m1.887152432s"
Jul 29 15:46:36.939012 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:36.938941    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-118.ec2.internal\" not found"
Jul 29 15:46:37.039142 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:37.039115    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-118.ec2.internal\" not found"
Jul 29 15:46:37.070014 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.069986    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:46:37.108178 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:37.108143    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf62d983a3713d310961fc20664891a3.slice/crio-1da89f4cc0e2c846084ce0ddf259784bcedf37d2184fb5f81566cfd4fca3b81c WatchSource:0}: Error finding container 1da89f4cc0e2c846084ce0ddf259784bcedf37d2184fb5f81566cfd4fca3b81c: Status 404 returned error can't find the container with id 1da89f4cc0e2c846084ce0ddf259784bcedf37d2184fb5f81566cfd4fca3b81c
Jul 29 15:46:37.112951 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.112931    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 15:46:37.118658 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.118638    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:46:37.126684 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:37.126660    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod334f61d991df88637e87bb21bfbae838.slice/crio-41515c00ada8c694101ef6846341710fb6ff538461c3d48d8ea5afc83fbd0b3d WatchSource:0}: Error finding container 41515c00ada8c694101ef6846341710fb6ff538461c3d48d8ea5afc83fbd0b3d: Status 404 returned error can't find the container with id 41515c00ada8c694101ef6846341710fb6ff538461c3d48d8ea5afc83fbd0b3d
Jul 29 15:46:37.143812 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.143792    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:37.157110 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.157086    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 15:46:37.157901 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.157887    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-118.ec2.internal"
Jul 29 15:46:37.171733 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.171713    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 15:46:37.638482 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.638450    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:46:37.822884 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.822853    2560 apiserver.go:52] "Watching apiserver"
Jul 29 15:46:37.832667 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.832642    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 15:46:37.833200 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.833128    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-118.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b","openshift-image-registry/node-ca-5l247","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal","openshift-multus/multus-additional-cni-plugins-pbksq","openshift-multus/multus-kl75q","openshift-multus/network-metrics-daemon-grvsv","openshift-ovn-kubernetes/ovnkube-node-bbndf","kube-system/global-pull-secret-syncer-6w8mg","kube-system/konnectivity-agent-rghwf","openshift-cluster-node-tuning-operator/tuned-bkp88","openshift-dns/node-resolver-kgn8d","openshift-network-diagnostics/network-check-target-z6jwx","openshift-network-operator/iptables-alerter-wbszn"]
Jul 29 15:46:37.834541 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.834522    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rghwf"
Jul 29 15:46:37.835761 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.835718    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:37.835853 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:37.835787    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:46:37.837530 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.837512    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5l247"
Jul 29 15:46:37.838457 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.838314    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 15:46:37.839033 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.838691    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-j7lrt\""
Jul 29 15:46:37.839033 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.838884    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 15:46:37.840127 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.840108    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wbszn"
Jul 29 15:46:37.840466 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.840440    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 15:46:37.840652 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.840634    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 15:46:37.840878 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.840858    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 15:46:37.841899 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.841877    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6q7s6\""
Jul 29 15:46:37.843097 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.842982    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.843488 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.843470    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 15:46:37.844385 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.843987    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.846180 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.845577    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:37.846180 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:37.845647    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:46:37.847092 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.847011    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:37.847092 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:37.847074    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:46:37.847236 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.847219    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 15:46:37.847420 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.847404    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 15:46:37.847531 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.847410    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 15:46:37.852203 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.852181    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dsstd\""
Jul 29 15:46:37.853264 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853245    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:37.853459 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853440    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6nln2\""
Jul 29 15:46:37.853556 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853538    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/44f4de5c-8d17-4b70-a306-0d470121cad5-iptables-alerter-script\") pod \"iptables-alerter-wbszn\" (UID: \"44f4de5c-8d17-4b70-a306-0d470121cad5\") " pod="openshift-network-operator/iptables-alerter-wbszn"
Jul 29 15:46:37.853618 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853601    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44f4de5c-8d17-4b70-a306-0d470121cad5-host-slash\") pod \"iptables-alerter-wbszn\" (UID: \"44f4de5c-8d17-4b70-a306-0d470121cad5\") " pod="openshift-network-operator/iptables-alerter-wbszn"
Jul 29 15:46:37.853670 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853628    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eeeada33-be3e-45ce-b580-86be8ee80c48-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.853742 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853716    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f5c78df-f246-43b5-a4c2-7704c6af4c1d-host\") pod \"node-ca-5l247\" (UID: \"5f5c78df-f246-43b5-a4c2-7704c6af4c1d\") " pod="openshift-image-registry/node-ca-5l247"
Jul 29 15:46:37.853797 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853749    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt77c\" (UniqueName: \"kubernetes.io/projected/5f5c78df-f246-43b5-a4c2-7704c6af4c1d-kube-api-access-dt77c\") pod \"node-ca-5l247\" (UID: \"5f5c78df-f246-43b5-a4c2-7704c6af4c1d\") " pod="openshift-image-registry/node-ca-5l247"
Jul 29 15:46:37.853854 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853799    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e456ec98-497c-4469-b9cb-d8f8724e044b-cni-binary-copy\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.853854 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853828    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-multus-socket-dir-parent\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.853854 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853844    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 15:46:37.854039 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853858    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e456ec98-497c-4469-b9cb-d8f8724e044b-multus-daemon-config\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.854039 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853889    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1-agent-certs\") pod \"konnectivity-agent-rghwf\" (UID: \"78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1\") " pod="kube-system/konnectivity-agent-rghwf"
Jul 29 15:46:37.854039 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853890    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 15:46:37.854039 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853941    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f5c78df-f246-43b5-a4c2-7704c6af4c1d-serviceca\") pod \"node-ca-5l247\" (UID: \"5f5c78df-f246-43b5-a4c2-7704c6af4c1d\") " pod="openshift-image-registry/node-ca-5l247"
Jul 29 15:46:37.854039 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.853984    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bll4r\" (UniqueName: \"kubernetes.io/projected/eeeada33-be3e-45ce-b580-86be8ee80c48-kube-api-access-bll4r\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.854039 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854012    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-run-netns\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.854310 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854041    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-hostroot\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.854310 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854067    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-multus-conf-dir\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.854310 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854100    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-run-multus-certs\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.854310 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854128    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-etc-kubernetes\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.854310 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854142    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 15:46:37.854310 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854152    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eeeada33-be3e-45ce-b580-86be8ee80c48-system-cni-dir\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.854310 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854184    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eeeada33-be3e-45ce-b580-86be8ee80c48-os-release\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.854310 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854225    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-var-lib-cni-bin\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.854310 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854255    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-var-lib-cni-multus\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.854310 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854283    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xclg8\" (UniqueName: \"kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8\") pod \"network-check-target-z6jwx\" (UID: \"c45ef580-96eb-4119-89d3-d4dd468e7917\") " pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:37.854766 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854319    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-multus-cni-dir\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.854766 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854344    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tdnm\" (UniqueName: \"kubernetes.io/projected/e456ec98-497c-4469-b9cb-d8f8724e044b-kube-api-access-5tdnm\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.854766 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854380    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vtt8\" (UniqueName: \"kubernetes.io/projected/44f4de5c-8d17-4b70-a306-0d470121cad5-kube-api-access-6vtt8\") pod \"iptables-alerter-wbszn\" (UID: \"44f4de5c-8d17-4b70-a306-0d470121cad5\") " pod="openshift-network-operator/iptables-alerter-wbszn"
Jul 29 15:46:37.854766 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854429    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eeeada33-be3e-45ce-b580-86be8ee80c48-cnibin\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.854766 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854562    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eeeada33-be3e-45ce-b580-86be8ee80c48-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.854766 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854597    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-os-release\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.854766 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854648    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-run-k8s-cni-cncf-io\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.854766 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854734    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eeeada33-be3e-45ce-b580-86be8ee80c48-cni-binary-copy\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.855205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854798    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-var-lib-kubelet\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.855205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854841    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eeeada33-be3e-45ce-b580-86be8ee80c48-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.855205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854883    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-system-cni-dir\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.855205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854909    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-cnibin\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.855205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854960    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1-konnectivity-ca\") pod \"konnectivity-agent-rghwf\" (UID: \"78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1\") " pod="kube-system/konnectivity-agent-rghwf"
Jul 29 15:46:37.855205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.854973    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8dnqm\""
Jul 29 15:46:37.855205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.855038    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 15:46:37.855205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.855082    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 15:46:37.855553 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.855321    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 15:46:37.857049 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.857027    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 15:46:37.858126 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.857398    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.858126 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.857720    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 15:46:37.858126 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.858041    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h2xh6\""
Jul 29 15:46:37.858471 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.858452    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 15:46:37.859215 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.859198    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.860348 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.860329    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 15:46:37.860739 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.860580    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 15:46:37.861725 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.861715    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kgn8d"
Jul 29 15:46:37.863123 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.863106    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 15:46:37.863486 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.863220    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 15:46:37.863757 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.863275    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 15:46:37.864526 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.864510    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 15:46:37.864872 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.864855    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-q8279\""
Jul 29 15:46:37.865324 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.865303    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 15:46:37.865542 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.865526    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-blxnj\""
Jul 29 15:46:37.865888 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.865870    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 15:46:37.866079 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.866066    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 15:46:37.866845 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.866826    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 15:46:37.867039 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.867025    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-b5j8b\""
Jul 29 15:46:37.876289 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.876271    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 15:46:37.893754 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.893703    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hj5bw"
Jul 29 15:46:37.903304 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.902980    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hj5bw"
Jul 29 15:46:37.944552 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.944527    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 15:46:37.955300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955271    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-system-cni-dir\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.955405 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955314    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1-konnectivity-ca\") pod \"konnectivity-agent-rghwf\" (UID: \"78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1\") " pod="kube-system/konnectivity-agent-rghwf"
Jul 29 15:46:37.955405 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955340    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-sysconfig\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.955405 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955364    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-lib-modules\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.955405 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955385    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-slash\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.955405 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955393    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-system-cni-dir\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.955645 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955410    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/be69d266-eb3a-43bd-b8be-cceea4eda917-ovn-node-metrics-cert\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.955645 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955440    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/44f4de5c-8d17-4b70-a306-0d470121cad5-iptables-alerter-script\") pod \"iptables-alerter-wbszn\" (UID: \"44f4de5c-8d17-4b70-a306-0d470121cad5\") " pod="openshift-network-operator/iptables-alerter-wbszn"
Jul 29 15:46:37.955645 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955479    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44f4de5c-8d17-4b70-a306-0d470121cad5-host-slash\") pod \"iptables-alerter-wbszn\" (UID: \"44f4de5c-8d17-4b70-a306-0d470121cad5\") " pod="openshift-network-operator/iptables-alerter-wbszn"
Jul 29 15:46:37.955645 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955501    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-tmp\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.955645 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955526    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt77c\" (UniqueName: \"kubernetes.io/projected/5f5c78df-f246-43b5-a4c2-7704c6af4c1d-kube-api-access-dt77c\") pod \"node-ca-5l247\" (UID: \"5f5c78df-f246-43b5-a4c2-7704c6af4c1d\") " pod="openshift-image-registry/node-ca-5l247"
Jul 29 15:46:37.955645 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955549    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e456ec98-497c-4469-b9cb-d8f8724e044b-cni-binary-copy\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.955645 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955560    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44f4de5c-8d17-4b70-a306-0d470121cad5-host-slash\") pod \"iptables-alerter-wbszn\" (UID: \"44f4de5c-8d17-4b70-a306-0d470121cad5\") " pod="openshift-network-operator/iptables-alerter-wbszn"
Jul 29 15:46:37.955645 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955572    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-multus-socket-dir-parent\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.955645 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955619    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e456ec98-497c-4469-b9cb-d8f8724e044b-multus-daemon-config\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.956082 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955648    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-systemd\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.956082 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955674    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/be69d266-eb3a-43bd-b8be-cceea4eda917-ovnkube-config\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.956082 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955704    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bll4r\" (UniqueName: \"kubernetes.io/projected/eeeada33-be3e-45ce-b580-86be8ee80c48-kube-api-access-bll4r\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.956082 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955727    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xclg8\" (UniqueName: \"kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8\") pod \"network-check-target-z6jwx\" (UID: \"c45ef580-96eb-4119-89d3-d4dd468e7917\") " pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:37.956082 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955731    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-multus-socket-dir-parent\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.956082 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955748    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eeeada33-be3e-45ce-b580-86be8ee80c48-system-cni-dir\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.956082 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.955775    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-var-lib-cni-bin\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.956082 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956033    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1-konnectivity-ca\") pod \"konnectivity-agent-rghwf\" (UID: \"78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1\") " pod="kube-system/konnectivity-agent-rghwf"
Jul 29 15:46:37.956082 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956072    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/44f4de5c-8d17-4b70-a306-0d470121cad5-iptables-alerter-script\") pod \"iptables-alerter-wbszn\" (UID: \"44f4de5c-8d17-4b70-a306-0d470121cad5\") " pod="openshift-network-operator/iptables-alerter-wbszn"
Jul 29 15:46:37.956494 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956140    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-var-lib-cni-bin\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.956494 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956290    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-var-lib-cni-multus\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.956494 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956327    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-registration-dir\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:37.956494 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956363    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-etc-openvswitch\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.956494 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956451    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/be69d266-eb3a-43bd-b8be-cceea4eda917-env-overrides\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.956494 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956460    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eeeada33-be3e-45ce-b580-86be8ee80c48-system-cni-dir\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.956494 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956464    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-var-lib-cni-multus\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.956822 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956523    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-sysctl-d\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.956822 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956557    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-host\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.956822 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956586    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhdff\" (UniqueName: \"kubernetes.io/projected/be69d266-eb3a-43bd-b8be-cceea4eda917-kube-api-access-lhdff\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.956822 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956628    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e456ec98-497c-4469-b9cb-d8f8724e044b-multus-daemon-config\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.956822 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956672    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eeeada33-be3e-45ce-b580-86be8ee80c48-cnibin\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.956822 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956715    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:37.956822 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956771    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-node-log\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.956822 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956800    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eeeada33-be3e-45ce-b580-86be8ee80c48-cnibin\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.957222 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956837    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-tuned\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.957222 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956958    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e456ec98-497c-4469-b9cb-d8f8724e044b-cni-binary-copy\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.957222 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.956962    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-cni-netd\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.957222 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957022    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eeeada33-be3e-45ce-b580-86be8ee80c48-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.957222 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957071    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-cnibin\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.957222 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957106    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-sysctl-conf\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.957222 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957134    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-sys\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.957222 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957166    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-run-systemd\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.957222 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957188    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/be69d266-eb3a-43bd-b8be-cceea4eda917-ovnkube-script-lib\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.957222 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957213    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eeeada33-be3e-45ce-b580-86be8ee80c48-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.957685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957244    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-var-lib-kubelet\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.957685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957266    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6fde166c-e320-4357-8289-4450abb2cc06-hosts-file\") pod \"node-resolver-kgn8d\" (UID: \"6fde166c-e320-4357-8289-4450abb2cc06\") " pod="openshift-dns/node-resolver-kgn8d"
Jul 29 15:46:37.957685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957288    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbvh2\" (UniqueName: \"kubernetes.io/projected/6fde166c-e320-4357-8289-4450abb2cc06-kube-api-access-pbvh2\") pod \"node-resolver-kgn8d\" (UID: \"6fde166c-e320-4357-8289-4450abb2cc06\") " pod="openshift-dns/node-resolver-kgn8d"
Jul 29 15:46:37.957685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957317    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-sys-fs\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:37.957685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957354    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-log-socket\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.957685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957390    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-run-ovn-kubernetes\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.957685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957444    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-cnibin\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.957685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957570    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f5c78df-f246-43b5-a4c2-7704c6af4c1d-host\") pod \"node-ca-5l247\" (UID: \"5f5c78df-f246-43b5-a4c2-7704c6af4c1d\") " pod="openshift-image-registry/node-ca-5l247"
Jul 29 15:46:37.957685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957607    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1-agent-certs\") pod \"konnectivity-agent-rghwf\" (UID: \"78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1\") " pod="kube-system/konnectivity-agent-rghwf"
Jul 29 15:46:37.957685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957684    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pscv2\" (UniqueName: \"kubernetes.io/projected/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-kube-api-access-pscv2\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.958214 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957692    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eeeada33-be3e-45ce-b580-86be8ee80c48-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.958214 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957725    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-kubelet-config\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:37.958214 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957691    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f5c78df-f246-43b5-a4c2-7704c6af4c1d-host\") pod \"node-ca-5l247\" (UID: \"5f5c78df-f246-43b5-a4c2-7704c6af4c1d\") " pod="openshift-image-registry/node-ca-5l247"
Jul 29 15:46:37.958214 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957777    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxtzf\" (UniqueName: \"kubernetes.io/projected/902d7a09-dd0e-466d-95e3-a80d69c1dd11-kube-api-access-wxtzf\") pod \"network-metrics-daemon-grvsv\" (UID: \"902d7a09-dd0e-466d-95e3-a80d69c1dd11\") " pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:37.958214 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957827    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-socket-dir\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:37.958214 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957892    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eeeada33-be3e-45ce-b580-86be8ee80c48-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.958214 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.957893    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 15:46:37.958544 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958327    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-cni-bin\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.958544 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958393    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f5c78df-f246-43b5-a4c2-7704c6af4c1d-serviceca\") pod \"node-ca-5l247\" (UID: \"5f5c78df-f246-43b5-a4c2-7704c6af4c1d\") " pod="openshift-image-registry/node-ca-5l247"
Jul 29 15:46:37.958544 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958422    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-run-netns\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.958544 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958444    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-hostroot\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.958544 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958468    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-multus-conf-dir\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.958544 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958493    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-run-multus-certs\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.958544 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958514    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-etc-kubernetes\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.958544 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958542    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958566    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-etc-selinux\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958590    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eeeada33-be3e-45ce-b580-86be8ee80c48-os-release\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958613    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-modprobe-d\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958634    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-run\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958660    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6sp5\" (UniqueName: \"kubernetes.io/projected/5ad492c5-f51e-4656-83ee-ab9b28add71c-kube-api-access-k6sp5\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958683    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-kubelet\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958706    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-multus-cni-dir\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958730    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tdnm\" (UniqueName: \"kubernetes.io/projected/e456ec98-497c-4469-b9cb-d8f8724e044b-kube-api-access-5tdnm\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958747    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fde166c-e320-4357-8289-4450abb2cc06-tmp-dir\") pod \"node-resolver-kgn8d\" (UID: \"6fde166c-e320-4357-8289-4450abb2cc06\") " pod="openshift-dns/node-resolver-kgn8d"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958760    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-dbus\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958775    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs\") pod \"network-metrics-daemon-grvsv\" (UID: \"902d7a09-dd0e-466d-95e3-a80d69c1dd11\") " pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958795    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-device-dir\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958809    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-systemd-units\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958830    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-run-netns\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958854    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vtt8\" (UniqueName: \"kubernetes.io/projected/44f4de5c-8d17-4b70-a306-0d470121cad5-kube-api-access-6vtt8\") pod \"iptables-alerter-wbszn\" (UID: \"44f4de5c-8d17-4b70-a306-0d470121cad5\") " pod="openshift-network-operator/iptables-alerter-wbszn"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958870    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eeeada33-be3e-45ce-b580-86be8ee80c48-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.958896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958893    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-os-release\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.959692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958909    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-run-k8s-cni-cncf-io\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.959692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958939    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-kubernetes\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:37.959692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958953    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-var-lib-openvswitch\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.959692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958980    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-run-openvswitch\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.959692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.958996    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.959692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959013    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eeeada33-be3e-45ce-b580-86be8ee80c48-cni-binary-copy\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.959692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959029    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-var-lib-kubelet\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.959692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959042    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-run-ovn\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:37.959692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959354    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f5c78df-f246-43b5-a4c2-7704c6af4c1d-serviceca\") pod \"node-ca-5l247\" (UID: \"5f5c78df-f246-43b5-a4c2-7704c6af4c1d\") " pod="openshift-image-registry/node-ca-5l247"
Jul 29 15:46:37.959692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959404    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-run-netns\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.959692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959468    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-hostroot\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.959692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959576    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-multus-cni-dir\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.959692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959623    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eeeada33-be3e-45ce-b580-86be8ee80c48-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.959692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959676    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-os-release\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.960344 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959709    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-run-k8s-cni-cncf-io\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.960344 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959767    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-var-lib-kubelet\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.960344 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959802    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-multus-conf-dir\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.960344 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959836    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-host-run-multus-certs\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.960344 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959854    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e456ec98-497c-4469-b9cb-d8f8724e044b-etc-kubernetes\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.960344 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.959893    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eeeada33-be3e-45ce-b580-86be8ee80c48-os-release\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.960344 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.960202    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eeeada33-be3e-45ce-b580-86be8ee80c48-cni-binary-copy\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.963184 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.963145    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1-agent-certs\") pod \"konnectivity-agent-rghwf\" (UID: \"78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1\") " pod="kube-system/konnectivity-agent-rghwf"
Jul 29 15:46:37.969990 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:37.969966    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:46:37.969990 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:37.969990    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:46:37.970136 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:37.970002    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xclg8 for pod openshift-network-diagnostics/network-check-target-z6jwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:46:37.970136 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:37.970064    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8 podName:c45ef580-96eb-4119-89d3-d4dd468e7917 nodeName:}" failed. No retries permitted until 2026-07-29 15:46:38.470044837 +0000 UTC m=+2.912251078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xclg8" (UniqueName: "kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8") pod "network-check-target-z6jwx" (UID: "c45ef580-96eb-4119-89d3-d4dd468e7917") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:46:37.975632 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.975611    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bll4r\" (UniqueName: \"kubernetes.io/projected/eeeada33-be3e-45ce-b580-86be8ee80c48-kube-api-access-bll4r\") pod \"multus-additional-cni-plugins-pbksq\" (UID: \"eeeada33-be3e-45ce-b580-86be8ee80c48\") " pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:37.978365 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.978339    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt77c\" (UniqueName: \"kubernetes.io/projected/5f5c78df-f246-43b5-a4c2-7704c6af4c1d-kube-api-access-dt77c\") pod \"node-ca-5l247\" (UID: \"5f5c78df-f246-43b5-a4c2-7704c6af4c1d\") " pod="openshift-image-registry/node-ca-5l247"
Jul 29 15:46:37.986836 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.986815    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tdnm\" (UniqueName: \"kubernetes.io/projected/e456ec98-497c-4469-b9cb-d8f8724e044b-kube-api-access-5tdnm\") pod \"multus-kl75q\" (UID: \"e456ec98-497c-4469-b9cb-d8f8724e044b\") " pod="openshift-multus/multus-kl75q"
Jul 29 15:46:37.991379 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:37.991357    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vtt8\" (UniqueName: \"kubernetes.io/projected/44f4de5c-8d17-4b70-a306-0d470121cad5-kube-api-access-6vtt8\") pod \"iptables-alerter-wbszn\" (UID: \"44f4de5c-8d17-4b70-a306-0d470121cad5\") " pod="openshift-network-operator/iptables-alerter-wbszn"
Jul 29 15:46:38.041461 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.041405    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-118.ec2.internal" event={"ID":"cf62d983a3713d310961fc20664891a3","Type":"ContainerStarted","Data":"1da89f4cc0e2c846084ce0ddf259784bcedf37d2184fb5f81566cfd4fca3b81c"}
Jul 29 15:46:38.042587 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.042560    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal" event={"ID":"334f61d991df88637e87bb21bfbae838","Type":"ContainerStarted","Data":"41515c00ada8c694101ef6846341710fb6ff538461c3d48d8ea5afc83fbd0b3d"}
Jul 29 15:46:38.059369 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059339    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.059523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059383    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-etc-selinux\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.059523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059409    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-modprobe-d\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.059523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059434    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-run\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.059523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059458    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6sp5\" (UniqueName: \"kubernetes.io/projected/5ad492c5-f51e-4656-83ee-ab9b28add71c-kube-api-access-k6sp5\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.059523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059481    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-kubelet\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.059523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059465    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.059523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059509    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fde166c-e320-4357-8289-4450abb2cc06-tmp-dir\") pod \"node-resolver-kgn8d\" (UID: \"6fde166c-e320-4357-8289-4450abb2cc06\") " pod="openshift-dns/node-resolver-kgn8d"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059531    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-dbus\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059554    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-run\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059556    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs\") pod \"network-metrics-daemon-grvsv\" (UID: \"902d7a09-dd0e-466d-95e3-a80d69c1dd11\") " pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059599    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-device-dir\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059624    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-systemd-units\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:38.059635    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059650    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-run-netns\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059679    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-kubernetes\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:38.059695    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs podName:902d7a09-dd0e-466d-95e3-a80d69c1dd11 nodeName:}" failed. No retries permitted until 2026-07-29 15:46:38.559676465 +0000 UTC m=+3.001882717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs") pod "network-metrics-daemon-grvsv" (UID: "902d7a09-dd0e-466d-95e3-a80d69c1dd11") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059720    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-var-lib-openvswitch\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059752    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-run-openvswitch\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059769    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059784    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-dbus\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059795    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-run-ovn\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059816    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-sysconfig\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059722    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-kubernetes\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.059860 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059838    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-lib-modules\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059848    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-etc-selinux\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059856    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-slash\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059827    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-kubelet\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059867    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059872    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/be69d266-eb3a-43bd-b8be-cceea4eda917-ovn-node-metrics-cert\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059936    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-tmp\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059967    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-systemd\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059964    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-device-dir\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059889    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-var-lib-openvswitch\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.059990    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/be69d266-eb3a-43bd-b8be-cceea4eda917-ovnkube-config\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060031    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-registration-dir\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060029    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-systemd\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060083    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-registration-dir\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060130    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-sysconfig\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060182    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-systemd-units\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060217    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-run-openvswitch\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.060685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060251    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-run-ovn\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060258    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-lib-modules\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060282    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-run-netns\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060338    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-slash\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060353    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-modprobe-d\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060377    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-etc-openvswitch\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060407    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/be69d266-eb3a-43bd-b8be-cceea4eda917-env-overrides\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060434    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-sysctl-d\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060456    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-host\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060466    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-etc-openvswitch\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060480    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhdff\" (UniqueName: \"kubernetes.io/projected/be69d266-eb3a-43bd-b8be-cceea4eda917-kube-api-access-lhdff\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060507    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060530    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-node-log\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060535    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/be69d266-eb3a-43bd-b8be-cceea4eda917-ovnkube-config\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060552    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-tuned\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060575    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-cni-netd\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060583    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-sysctl-d\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060600    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-sysctl-conf\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.061536 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060605    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fde166c-e320-4357-8289-4450abb2cc06-tmp-dir\") pod \"node-resolver-kgn8d\" (UID: \"6fde166c-e320-4357-8289-4450abb2cc06\") " pod="openshift-dns/node-resolver-kgn8d"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060625    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-sys\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060647    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-run-systemd\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:38.060656    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060677    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-cni-netd\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:38.060714    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret podName:fce7281a-8d9e-4cda-a8ba-7480f20da3ef nodeName:}" failed. No retries permitted until 2026-07-29 15:46:38.560697166 +0000 UTC m=+3.002903402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret") pod "global-pull-secret-syncer-6w8mg" (UID: "fce7281a-8d9e-4cda-a8ba-7480f20da3ef") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060754    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-host\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060806    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-sysctl-conf\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060853    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/be69d266-eb3a-43bd-b8be-cceea4eda917-env-overrides\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060858    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-sys\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060893    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-run-systemd\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060680    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/be69d266-eb3a-43bd-b8be-cceea4eda917-ovnkube-script-lib\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060950    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-var-lib-kubelet\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060974    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6fde166c-e320-4357-8289-4450abb2cc06-hosts-file\") pod \"node-resolver-kgn8d\" (UID: \"6fde166c-e320-4357-8289-4450abb2cc06\") " pod="openshift-dns/node-resolver-kgn8d"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.060997    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbvh2\" (UniqueName: \"kubernetes.io/projected/6fde166c-e320-4357-8289-4450abb2cc06-kube-api-access-pbvh2\") pod \"node-resolver-kgn8d\" (UID: \"6fde166c-e320-4357-8289-4450abb2cc06\") " pod="openshift-dns/node-resolver-kgn8d"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061024    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-sys-fs\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061048    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-log-socket\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.062408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061074    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-run-ovn-kubernetes\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061107    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-node-log\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061123    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-run-ovn-kubernetes\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061170    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-log-socket\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061177    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-sys-fs\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061192    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/be69d266-eb3a-43bd-b8be-cceea4eda917-ovnkube-script-lib\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061223    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-var-lib-kubelet\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061229    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6fde166c-e320-4357-8289-4450abb2cc06-hosts-file\") pod \"node-resolver-kgn8d\" (UID: \"6fde166c-e320-4357-8289-4450abb2cc06\") " pod="openshift-dns/node-resolver-kgn8d"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061251    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pscv2\" (UniqueName: \"kubernetes.io/projected/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-kube-api-access-pscv2\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061280    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-kubelet-config\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061304    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxtzf\" (UniqueName: \"kubernetes.io/projected/902d7a09-dd0e-466d-95e3-a80d69c1dd11-kube-api-access-wxtzf\") pod \"network-metrics-daemon-grvsv\" (UID: \"902d7a09-dd0e-466d-95e3-a80d69c1dd11\") " pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061328    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-socket-dir\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061349    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-kubelet-config\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061355    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-cni-bin\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061397    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be69d266-eb3a-43bd-b8be-cceea4eda917-host-cni-bin\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.061485    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5ad492c5-f51e-4656-83ee-ab9b28add71c-socket-dir\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.062653    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/be69d266-eb3a-43bd-b8be-cceea4eda917-ovn-node-metrics-cert\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.063300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.063176    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-tmp\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.064035 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.063637    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-etc-tuned\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.076463 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.076436    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pscv2\" (UniqueName: \"kubernetes.io/projected/40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3-kube-api-access-pscv2\") pod \"tuned-bkp88\" (UID: \"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3\") " pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.076605 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.076516    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhdff\" (UniqueName: \"kubernetes.io/projected/be69d266-eb3a-43bd-b8be-cceea4eda917-kube-api-access-lhdff\") pod \"ovnkube-node-bbndf\" (UID: \"be69d266-eb3a-43bd-b8be-cceea4eda917\") " pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.076965 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.076938    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbvh2\" (UniqueName: \"kubernetes.io/projected/6fde166c-e320-4357-8289-4450abb2cc06-kube-api-access-pbvh2\") pod \"node-resolver-kgn8d\" (UID: \"6fde166c-e320-4357-8289-4450abb2cc06\") " pod="openshift-dns/node-resolver-kgn8d"
Jul 29 15:46:38.078590 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.078546    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6sp5\" (UniqueName: \"kubernetes.io/projected/5ad492c5-f51e-4656-83ee-ab9b28add71c-kube-api-access-k6sp5\") pod \"aws-ebs-csi-driver-node-c6q7b\" (UID: \"5ad492c5-f51e-4656-83ee-ab9b28add71c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.078855 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.078829    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxtzf\" (UniqueName: \"kubernetes.io/projected/902d7a09-dd0e-466d-95e3-a80d69c1dd11-kube-api-access-wxtzf\") pod \"network-metrics-daemon-grvsv\" (UID: \"902d7a09-dd0e-466d-95e3-a80d69c1dd11\") " pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:38.153888 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.153814    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rghwf"
Jul 29 15:46:38.168834 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.168812    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5l247"
Jul 29 15:46:38.177012 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.176987    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wbszn"
Jul 29 15:46:38.183896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.183875    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pbksq"
Jul 29 15:46:38.192042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.192020    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kl75q"
Jul 29 15:46:38.198706 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.198685    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b"
Jul 29 15:46:38.206527 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.206507    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:46:38.213142 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.213122    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bkp88"
Jul 29 15:46:38.220785 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.220769    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kgn8d"
Jul 29 15:46:38.565218 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.565137    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs\") pod \"network-metrics-daemon-grvsv\" (UID: \"902d7a09-dd0e-466d-95e3-a80d69c1dd11\") " pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:38.565382 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.565217    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xclg8\" (UniqueName: \"kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8\") pod \"network-check-target-z6jwx\" (UID: \"c45ef580-96eb-4119-89d3-d4dd468e7917\") " pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:38.565382 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.565258    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:38.565382 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:38.565307    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:46:38.565537 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:38.565379    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:46:38.565537 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:38.565388    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs podName:902d7a09-dd0e-466d-95e3-a80d69c1dd11 nodeName:}" failed. No retries permitted until 2026-07-29 15:46:39.565368916 +0000 UTC m=+4.007575164 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs") pod "network-metrics-daemon-grvsv" (UID: "902d7a09-dd0e-466d-95e3-a80d69c1dd11") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:46:38.565537 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:38.565435    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret podName:fce7281a-8d9e-4cda-a8ba-7480f20da3ef nodeName:}" failed. No retries permitted until 2026-07-29 15:46:39.565418835 +0000 UTC m=+4.007625096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret") pod "global-pull-secret-syncer-6w8mg" (UID: "fce7281a-8d9e-4cda-a8ba-7480f20da3ef") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:46:38.565537 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:38.565381    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:46:38.565537 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:38.565467    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:46:38.565537 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:38.565480    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xclg8 for pod openshift-network-diagnostics/network-check-target-z6jwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:46:38.565804 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:38.565564    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8 podName:c45ef580-96eb-4119-89d3-d4dd468e7917 nodeName:}" failed. No retries permitted until 2026-07-29 15:46:39.565552161 +0000 UTC m=+4.007758401 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xclg8" (UniqueName: "kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8") pod "network-check-target-z6jwx" (UID: "c45ef580-96eb-4119-89d3-d4dd468e7917") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:46:38.904863 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.904778    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 15:41:37 +0000 UTC" deadline="2028-04-30 18:11:04.30248474 +0000 UTC"
Jul 29 15:46:38.904863 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.904808    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15386h24m25.39768199s"
Jul 29 15:46:38.968197 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:38.968167    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:46:39.037435 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:39.037400    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:39.037593 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:39.037464    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:39.037593 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:39.037564    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:46:39.037723 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:39.037700    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:46:39.137008 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:39.136975    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode456ec98_497c_4469_b9cb_d8f8724e044b.slice/crio-01408013e3701f77f51ab6d6a91d830dc7e7a7298d2411d1447c57c348aff70e WatchSource:0}: Error finding container 01408013e3701f77f51ab6d6a91d830dc7e7a7298d2411d1447c57c348aff70e: Status 404 returned error can't find the container with id 01408013e3701f77f51ab6d6a91d830dc7e7a7298d2411d1447c57c348aff70e
Jul 29 15:46:39.138644 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:39.138609    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e2ac3e_ac6f_4857_9dde_b152cf9c2cf3.slice/crio-af49745ca510c0144ca248c9d817b1ae2bd807da688124f8cf212cd52db66bc9 WatchSource:0}: Error finding container af49745ca510c0144ca248c9d817b1ae2bd807da688124f8cf212cd52db66bc9: Status 404 returned error can't find the container with id af49745ca510c0144ca248c9d817b1ae2bd807da688124f8cf212cd52db66bc9
Jul 29 15:46:39.139227 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:39.139105    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad492c5_f51e_4656_83ee_ab9b28add71c.slice/crio-1252716d6a8ce879dcd18ad2e329995b77bac7e27a2e50c3dcfd17aff3a34428 WatchSource:0}: Error finding container 1252716d6a8ce879dcd18ad2e329995b77bac7e27a2e50c3dcfd17aff3a34428: Status 404 returned error can't find the container with id 1252716d6a8ce879dcd18ad2e329995b77bac7e27a2e50c3dcfd17aff3a34428
Jul 29 15:46:39.140069 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:39.140046    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f4de5c_8d17_4b70_a306_0d470121cad5.slice/crio-ea6ccbdc61107e859cbbaf6f19e72a1b86041525adb5dc8fb1bd9384cb7bdac5 WatchSource:0}: Error finding container ea6ccbdc61107e859cbbaf6f19e72a1b86041525adb5dc8fb1bd9384cb7bdac5: Status 404 returned error can't find the container with id ea6ccbdc61107e859cbbaf6f19e72a1b86041525adb5dc8fb1bd9384cb7bdac5
Jul 29 15:46:39.141251 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:39.141233    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe69d266_eb3a_43bd_b8be_cceea4eda917.slice/crio-2c1eac1e0479f4f7ff86d699b5a515f733e44f504c7657d7ead9678a1de30dc5 WatchSource:0}: Error finding container 2c1eac1e0479f4f7ff86d699b5a515f733e44f504c7657d7ead9678a1de30dc5: Status 404 returned error can't find the container with id 2c1eac1e0479f4f7ff86d699b5a515f733e44f504c7657d7ead9678a1de30dc5
Jul 29 15:46:39.145749 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:39.145720    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeeada33_be3e_45ce_b580_86be8ee80c48.slice/crio-1cd9d921b115ed9a7871750f44860aceff1a2b4671a4e53b7b302714cc1de4c6 WatchSource:0}: Error finding container 1cd9d921b115ed9a7871750f44860aceff1a2b4671a4e53b7b302714cc1de4c6: Status 404 returned error can't find the container with id 1cd9d921b115ed9a7871750f44860aceff1a2b4671a4e53b7b302714cc1de4c6
Jul 29 15:46:39.147702 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:39.147681    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5c78df_f246_43b5_a4c2_7704c6af4c1d.slice/crio-4ba5a4c165ba9edcdd55e8d05ba01d8a6640e54009e8f84f1415a204494d53dc WatchSource:0}: Error finding container 4ba5a4c165ba9edcdd55e8d05ba01d8a6640e54009e8f84f1415a204494d53dc: Status 404 returned error can't find the container with id 4ba5a4c165ba9edcdd55e8d05ba01d8a6640e54009e8f84f1415a204494d53dc
Jul 29 15:46:39.148057 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:46:39.147980    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fde166c_e320_4357_8289_4450abb2cc06.slice/crio-16bafaaa0868a0f2708909043be0e9999dc98359b46c6ebb1e8bd67ca57fb482 WatchSource:0}: Error finding container 16bafaaa0868a0f2708909043be0e9999dc98359b46c6ebb1e8bd67ca57fb482: Status 404 returned error can't find the container with id 16bafaaa0868a0f2708909043be0e9999dc98359b46c6ebb1e8bd67ca57fb482
Jul 29 15:46:39.573607 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:39.573275    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs\") pod \"network-metrics-daemon-grvsv\" (UID: \"902d7a09-dd0e-466d-95e3-a80d69c1dd11\") " pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:39.573607 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:39.573569    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xclg8\" (UniqueName: \"kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8\") pod \"network-check-target-z6jwx\" (UID: \"c45ef580-96eb-4119-89d3-d4dd468e7917\") " pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:39.573607 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:39.573435    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:46:39.573607 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:39.573607    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:39.573945 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:39.573679    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs podName:902d7a09-dd0e-466d-95e3-a80d69c1dd11 nodeName:}" failed. No retries permitted until 2026-07-29 15:46:41.573654018 +0000 UTC m=+6.015860270 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs") pod "network-metrics-daemon-grvsv" (UID: "902d7a09-dd0e-466d-95e3-a80d69c1dd11") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:46:39.573945 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:39.573718    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:46:39.573945 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:39.573734    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:46:39.573945 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:39.573734    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:46:39.573945 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:39.573748    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xclg8 for pod openshift-network-diagnostics/network-check-target-z6jwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:46:39.573945 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:39.573785    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret podName:fce7281a-8d9e-4cda-a8ba-7480f20da3ef nodeName:}" failed. No retries permitted until 2026-07-29 15:46:41.573770096 +0000 UTC m=+6.015976335 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret") pod "global-pull-secret-syncer-6w8mg" (UID: "fce7281a-8d9e-4cda-a8ba-7480f20da3ef") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:46:39.573945 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:39.573804    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8 podName:c45ef580-96eb-4119-89d3-d4dd468e7917 nodeName:}" failed. No retries permitted until 2026-07-29 15:46:41.573795112 +0000 UTC m=+6.016001363 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xclg8" (UniqueName: "kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8") pod "network-check-target-z6jwx" (UID: "c45ef580-96eb-4119-89d3-d4dd468e7917") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:46:39.905474 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:39.905372    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 15:41:37 +0000 UTC" deadline="2028-05-12 10:18:57.914287254 +0000 UTC"
Jul 29 15:46:39.905474 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:39.905428    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15666h32m18.008862931s"
Jul 29 15:46:40.050953 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:40.050930    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:40.051109 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:40.051085    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:46:40.075408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:40.075376    2560 generic.go:356] "Generic (PLEG): container finished" podID="334f61d991df88637e87bb21bfbae838" containerID="8d4692de99b1e507697459b0aaf3266446ab20ba6b6841bba6e96096f1831c24" exitCode=0
Jul 29 15:46:40.075558 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:40.075461    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal" event={"ID":"334f61d991df88637e87bb21bfbae838","Type":"ContainerDied","Data":"8d4692de99b1e507697459b0aaf3266446ab20ba6b6841bba6e96096f1831c24"}
Jul 29 15:46:40.086836 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:40.086790    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf" event={"ID":"be69d266-eb3a-43bd-b8be-cceea4eda917","Type":"ContainerStarted","Data":"2c1eac1e0479f4f7ff86d699b5a515f733e44f504c7657d7ead9678a1de30dc5"}
Jul 29 15:46:40.098039 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:40.098011    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wbszn" event={"ID":"44f4de5c-8d17-4b70-a306-0d470121cad5","Type":"ContainerStarted","Data":"ea6ccbdc61107e859cbbaf6f19e72a1b86041525adb5dc8fb1bd9384cb7bdac5"}
Jul 29 15:46:40.122283 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:40.122260    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kl75q" event={"ID":"e456ec98-497c-4469-b9cb-d8f8724e044b","Type":"ContainerStarted","Data":"01408013e3701f77f51ab6d6a91d830dc7e7a7298d2411d1447c57c348aff70e"}
Jul 29 15:46:40.127475 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:40.127422    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rghwf" event={"ID":"78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1","Type":"ContainerStarted","Data":"a805840a42b72795d1171fcefdcbf3eaa1077c8aacf0d8cb5c897732d362c636"}
Jul 29 15:46:40.139963 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:40.139537    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-118.ec2.internal" event={"ID":"cf62d983a3713d310961fc20664891a3","Type":"ContainerStarted","Data":"02d37af3966c1864a6f3e95bc5ea0a55cbe62f412dc481e8f8683cae9af32e3e"}
Jul 29 15:46:40.145822 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:40.145740    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kgn8d" event={"ID":"6fde166c-e320-4357-8289-4450abb2cc06","Type":"ContainerStarted","Data":"16bafaaa0868a0f2708909043be0e9999dc98359b46c6ebb1e8bd67ca57fb482"}
Jul 29 15:46:40.151987 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:40.151889    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5l247" event={"ID":"5f5c78df-f246-43b5-a4c2-7704c6af4c1d","Type":"ContainerStarted","Data":"4ba5a4c165ba9edcdd55e8d05ba01d8a6640e54009e8f84f1415a204494d53dc"}
Jul 29 15:46:40.162562 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:40.161476    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-118.ec2.internal" podStartSLOduration=3.161459762 podStartE2EDuration="3.161459762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:46:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:46:40.161276717 +0000 UTC m=+4.603482977" watchObservedRunningTime="2026-07-29 15:46:40.161459762 +0000 UTC m=+4.603666023"
Jul 29 15:46:40.168155 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:40.168082    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pbksq" event={"ID":"eeeada33-be3e-45ce-b580-86be8ee80c48","Type":"ContainerStarted","Data":"1cd9d921b115ed9a7871750f44860aceff1a2b4671a4e53b7b302714cc1de4c6"}
Jul 29 15:46:40.170955 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:40.170866    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b" event={"ID":"5ad492c5-f51e-4656-83ee-ab9b28add71c","Type":"ContainerStarted","Data":"1252716d6a8ce879dcd18ad2e329995b77bac7e27a2e50c3dcfd17aff3a34428"}
Jul 29 15:46:40.180096 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:40.180048    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bkp88" event={"ID":"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3","Type":"ContainerStarted","Data":"af49745ca510c0144ca248c9d817b1ae2bd807da688124f8cf212cd52db66bc9"}
Jul 29 15:46:41.037666 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:41.037015    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:41.037666 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:41.037145    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:46:41.037666 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:41.037221    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:41.037666 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:41.037294    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:46:41.193487 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:41.193393    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal" event={"ID":"334f61d991df88637e87bb21bfbae838","Type":"ContainerStarted","Data":"bea5519809fb082d9ed5c89792e300c2b5560c5d98ed65116d557efaa415df33"}
Jul 29 15:46:41.591061 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:41.590168    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs\") pod \"network-metrics-daemon-grvsv\" (UID: \"902d7a09-dd0e-466d-95e3-a80d69c1dd11\") " pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:41.591061 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:41.590227    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xclg8\" (UniqueName: \"kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8\") pod \"network-check-target-z6jwx\" (UID: \"c45ef580-96eb-4119-89d3-d4dd468e7917\") " pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:41.591061 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:41.590262    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:41.591061 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:41.590398    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:46:41.591061 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:41.590453    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret podName:fce7281a-8d9e-4cda-a8ba-7480f20da3ef nodeName:}" failed. No retries permitted until 2026-07-29 15:46:45.590435886 +0000 UTC m=+10.032642126 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret") pod "global-pull-secret-syncer-6w8mg" (UID: "fce7281a-8d9e-4cda-a8ba-7480f20da3ef") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:46:41.591061 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:41.590818    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:46:41.591061 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:41.590865    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs podName:902d7a09-dd0e-466d-95e3-a80d69c1dd11 nodeName:}" failed. No retries permitted until 2026-07-29 15:46:45.590850115 +0000 UTC m=+10.033056353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs") pod "network-metrics-daemon-grvsv" (UID: "902d7a09-dd0e-466d-95e3-a80d69c1dd11") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:46:41.591061 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:41.590956    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:46:41.591061 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:41.590980    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:46:41.591061 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:41.590992    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xclg8 for pod openshift-network-diagnostics/network-check-target-z6jwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:46:41.591061 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:41.591030    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8 podName:c45ef580-96eb-4119-89d3-d4dd468e7917 nodeName:}" failed. No retries permitted until 2026-07-29 15:46:45.591017543 +0000 UTC m=+10.033223783 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xclg8" (UniqueName: "kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8") pod "network-check-target-z6jwx" (UID: "c45ef580-96eb-4119-89d3-d4dd468e7917") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:46:42.040672 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:42.039975    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:42.040672 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:42.040118    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:46:43.036657 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:43.036621    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:43.036848 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:43.036757    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:46:43.036848 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:43.036832    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:43.036976 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:43.036932    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:46:44.041130 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:44.040809    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:44.041130 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:44.040958    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:46:45.036883 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:45.036843    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:45.037064 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:45.036989    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:46:45.037145 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:45.037104    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:45.037235 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:45.037211    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:46:45.624345 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:45.624305    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:45.624811 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:45.624386    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs\") pod \"network-metrics-daemon-grvsv\" (UID: \"902d7a09-dd0e-466d-95e3-a80d69c1dd11\") " pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:45.624811 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:45.624427    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xclg8\" (UniqueName: \"kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8\") pod \"network-check-target-z6jwx\" (UID: \"c45ef580-96eb-4119-89d3-d4dd468e7917\") " pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:45.625192 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:45.624987    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:46:45.625192 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:45.624990    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:46:45.625192 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:45.625066    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret podName:fce7281a-8d9e-4cda-a8ba-7480f20da3ef nodeName:}" failed. No retries permitted until 2026-07-29 15:46:53.625050959 +0000 UTC m=+18.067257196 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret") pod "global-pull-secret-syncer-6w8mg" (UID: "fce7281a-8d9e-4cda-a8ba-7480f20da3ef") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:46:45.625192 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:45.625084    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs podName:902d7a09-dd0e-466d-95e3-a80d69c1dd11 nodeName:}" failed. No retries permitted until 2026-07-29 15:46:53.625076092 +0000 UTC m=+18.067282328 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs") pod "network-metrics-daemon-grvsv" (UID: "902d7a09-dd0e-466d-95e3-a80d69c1dd11") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:46:45.625192 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:45.625150    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:46:45.625192 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:45.625176    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:46:45.625192 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:45.625191    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xclg8 for pod openshift-network-diagnostics/network-check-target-z6jwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:46:45.625601 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:45.625257    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8 podName:c45ef580-96eb-4119-89d3-d4dd468e7917 nodeName:}" failed. No retries permitted until 2026-07-29 15:46:53.625239804 +0000 UTC m=+18.067446043 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xclg8" (UniqueName: "kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8") pod "network-check-target-z6jwx" (UID: "c45ef580-96eb-4119-89d3-d4dd468e7917") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:46:46.038720 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:46.038215    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:46.038720 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:46.038340    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:46:47.037419 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:47.037393    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:47.037419 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:47.037413    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:47.037911 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:47.037523    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:46:47.037911 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:47.037649    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:46:48.037177 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:48.037148    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:48.037384 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:48.037282    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:46:49.036897 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:49.036862    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:49.036897 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:49.036905    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:49.037390 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:49.037008    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:46:49.037390 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:49.037087    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:46:50.037314 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:50.037282    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:50.037773 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:50.037427    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:46:51.037140 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:51.037104    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:51.037312 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:51.037104    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:51.037312 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:51.037224    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:46:51.037728 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:51.037314    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:46:52.037080 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:52.037047    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:52.037242 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:52.037181    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:46:53.037233 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:53.037203    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:53.037740 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:53.037211    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:53.037740 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:53.037309    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:46:53.037740 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:53.037409    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:46:53.684666 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:53.684632    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs\") pod \"network-metrics-daemon-grvsv\" (UID: \"902d7a09-dd0e-466d-95e3-a80d69c1dd11\") " pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:53.684832 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:53.684675    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xclg8\" (UniqueName: \"kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8\") pod \"network-check-target-z6jwx\" (UID: \"c45ef580-96eb-4119-89d3-d4dd468e7917\") " pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:53.684832 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:53.684786    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:46:53.684832 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:53.684806    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:46:53.684832 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:53.684818    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xclg8 for pod openshift-network-diagnostics/network-check-target-z6jwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:46:53.684832 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:53.684829    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:46:53.685005 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:53.684872    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs podName:902d7a09-dd0e-466d-95e3-a80d69c1dd11 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:09.684853855 +0000 UTC m=+34.127060097 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs") pod "network-metrics-daemon-grvsv" (UID: "902d7a09-dd0e-466d-95e3-a80d69c1dd11") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:46:53.685005 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:53.684888    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8 podName:c45ef580-96eb-4119-89d3-d4dd468e7917 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:09.684879021 +0000 UTC m=+34.127085258 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xclg8" (UniqueName: "kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8") pod "network-check-target-z6jwx" (UID: "c45ef580-96eb-4119-89d3-d4dd468e7917") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:46:53.685005 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:53.684939    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:53.685110 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:53.685038    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:46:53.685110 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:53.685083    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret podName:fce7281a-8d9e-4cda-a8ba-7480f20da3ef nodeName:}" failed. No retries permitted until 2026-07-29 15:47:09.685071002 +0000 UTC m=+34.127277239 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret") pod "global-pull-secret-syncer-6w8mg" (UID: "fce7281a-8d9e-4cda-a8ba-7480f20da3ef") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:46:54.039864 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:54.039836    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:54.040282 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:54.039961    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:46:55.037069 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:55.037036    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:55.037259 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:55.037160    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:46:55.037259 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:55.037037    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:55.037374 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:55.037269    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:46:56.038040 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:56.038014    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:56.038487 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:56.038102    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:46:57.037207 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:57.037171    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:57.037382 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:57.037209    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:57.037382 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:57.037292    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:46:57.037481 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:57.037433    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:46:58.037349 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.037175    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:46:58.038114 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:58.037448    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:46:58.231297 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.231223    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf" event={"ID":"be69d266-eb3a-43bd-b8be-cceea4eda917","Type":"ContainerStarted","Data":"327daeec31a0386ad4b3b7feccfdaff6e13a3d44d017fbf93320440402457604"}
Jul 29 15:46:58.231448 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.231316    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf" event={"ID":"be69d266-eb3a-43bd-b8be-cceea4eda917","Type":"ContainerStarted","Data":"578d736aaefcc9d403580a2b70e785767fde4defc312de39334b97ba4952dada"}
Jul 29 15:46:58.231448 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.231367    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf" event={"ID":"be69d266-eb3a-43bd-b8be-cceea4eda917","Type":"ContainerStarted","Data":"05a6b8c5df01cb59eb4e386c6765a112f99eab63449e0945d4c55c234efc51e3"}
Jul 29 15:46:58.231448 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.231414    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf" event={"ID":"be69d266-eb3a-43bd-b8be-cceea4eda917","Type":"ContainerStarted","Data":"928320a211fd059b3209454f66f2ab2250f06b09cf5c33aa457dd1c73177a460"}
Jul 29 15:46:58.231596 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.231462    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf" event={"ID":"be69d266-eb3a-43bd-b8be-cceea4eda917","Type":"ContainerStarted","Data":"6b6725fd5f3492afdb921d10aa710895187568bee6b899b6f14512ac0e5eb145"}
Jul 29 15:46:58.231596 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.231507    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf" event={"ID":"be69d266-eb3a-43bd-b8be-cceea4eda917","Type":"ContainerStarted","Data":"6bcf02e633d96ae80162098eab2327ed00672d9d21548995cf789698a6b6113e"}
Jul 29 15:46:58.232871 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.232846    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kl75q" event={"ID":"e456ec98-497c-4469-b9cb-d8f8724e044b","Type":"ContainerStarted","Data":"72c0a3b9c9aa90935798d81f78b663c6011f843586e85a201de474a43c116d1e"}
Jul 29 15:46:58.234384 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.234362    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rghwf" event={"ID":"78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1","Type":"ContainerStarted","Data":"ec524b3b1b906fd4b274b2dfa0ccb5a80881a5d9edcaf5d4c2be9fb18b522c29"}
Jul 29 15:46:58.235772 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.235751    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kgn8d" event={"ID":"6fde166c-e320-4357-8289-4450abb2cc06","Type":"ContainerStarted","Data":"6f44865b143a57ca92d4944493419cb43227c03571f96ff131ed1c9ae03f885e"}
Jul 29 15:46:58.237219 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.237195    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5l247" event={"ID":"5f5c78df-f246-43b5-a4c2-7704c6af4c1d","Type":"ContainerStarted","Data":"e25f3edf85fc85d083c2c14891ac31d817c0b6c2902a88b19d8d552db9e9c96b"}
Jul 29 15:46:58.238858 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.238835    2560 generic.go:356] "Generic (PLEG): container finished" podID="eeeada33-be3e-45ce-b580-86be8ee80c48" containerID="7acc9acadb1b6a87cc46452ac33a984cf7e7ea348567531801a2e4b6447a8c47" exitCode=0
Jul 29 15:46:58.238958 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.238930    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pbksq" event={"ID":"eeeada33-be3e-45ce-b580-86be8ee80c48","Type":"ContainerDied","Data":"7acc9acadb1b6a87cc46452ac33a984cf7e7ea348567531801a2e4b6447a8c47"}
Jul 29 15:46:58.240501 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.240474    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b" event={"ID":"5ad492c5-f51e-4656-83ee-ab9b28add71c","Type":"ContainerStarted","Data":"0240ed06677f35a9cce6bf9fe78fa86cdf187f0bc87c45fec2d78dda82f05242"}
Jul 29 15:46:58.242023 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.242001    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bkp88" event={"ID":"40e2ac3e-ac6f-4857-9dde-b152cf9c2cf3","Type":"ContainerStarted","Data":"c08088e935fb800ca740f37bb4f36e54737db2d303ebeb9ad1491ddef76b0c0a"}
Jul 29 15:46:58.253666 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.253390    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-118.ec2.internal" podStartSLOduration=21.253373336 podStartE2EDuration="21.253373336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:46:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:46:41.212981312 +0000 UTC m=+5.655187572" watchObservedRunningTime="2026-07-29 15:46:58.253373336 +0000 UTC m=+22.695579596"
Jul 29 15:46:58.253795 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.253636    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kl75q" podStartSLOduration=3.896175189 podStartE2EDuration="22.253626845s" totalImagesPullingTime="18.357451656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:46:58.252275457 +0000 UTC m=+22.694481719" watchObservedRunningTime="2026-07-29 15:46:58.253626845 +0000 UTC m=+22.695833106"
Jul 29 15:46:58.293483 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.293439    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rghwf" podStartSLOduration=7.884333688 podStartE2EDuration="22.29342437s" totalImagesPullingTime="14.409090682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:46:58.292930485 +0000 UTC m=+22.735136740" watchObservedRunningTime="2026-07-29 15:46:58.29342437 +0000 UTC m=+22.735630632"
Jul 29 15:46:58.313270 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.313225    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bkp88" podStartSLOduration=4.012439198 podStartE2EDuration="22.313213552s" totalImagesPullingTime="18.300774354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:46:58.312880775 +0000 UTC m=+22.755087035" watchObservedRunningTime="2026-07-29 15:46:58.313213552 +0000 UTC m=+22.755419812"
Jul 29 15:46:58.329902 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.329851    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kgn8d" podStartSLOduration=4.037709116 podStartE2EDuration="22.329835964s" totalImagesPullingTime="18.292126848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:46:58.329034793 +0000 UTC m=+22.771241055" watchObservedRunningTime="2026-07-29 15:46:58.329835964 +0000 UTC m=+22.772042223"
Jul 29 15:46:58.347837 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.347799    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5l247" podStartSLOduration=4.055430182 podStartE2EDuration="22.347787274s" totalImagesPullingTime="18.292357092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:46:58.347191752 +0000 UTC m=+22.789398010" watchObservedRunningTime="2026-07-29 15:46:58.347787274 +0000 UTC m=+22.789993529"
Jul 29 15:46:58.756374 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.756353    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 15:46:58.921245 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.921114    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T15:46:58.756369351Z","UUID":"0ae1c9ff-e169-4a03-a748-eb80acffcad2","Handler":null,"Name":"","Endpoint":""}
Jul 29 15:46:58.922578 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.922559    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 15:46:58.922684 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:58.922593    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 15:46:59.037684 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:59.037651    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:46:59.038152 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:59.037746    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:46:59.038152 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:59.037803    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:46:59.038152 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:46:59.037880    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:46:59.247210 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:59.247123    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wbszn" event={"ID":"44f4de5c-8d17-4b70-a306-0d470121cad5","Type":"ContainerStarted","Data":"3c05e8bd361a93990d8709794a02bb5956ce735e6b2a4cb7331cdfaec605a430"}
Jul 29 15:46:59.248932 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:59.248897    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b" event={"ID":"5ad492c5-f51e-4656-83ee-ab9b28add71c","Type":"ContainerStarted","Data":"1f8424d3b5d94377a691c2419762fff0e7578f4bfb5d5886c6965aa8f9ce98b5"}
Jul 29 15:46:59.264783 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:46:59.264742    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wbszn" podStartSLOduration=4.968392036 podStartE2EDuration="23.264730317s" totalImagesPullingTime="18.296338281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:46:59.26406705 +0000 UTC m=+23.706273308" watchObservedRunningTime="2026-07-29 15:46:59.264730317 +0000 UTC m=+23.706936576"
Jul 29 15:47:00.036690 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:00.036654    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:47:00.036867 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:00.036772    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:47:01.037694 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:01.037658    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:47:01.038137 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:01.037700    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:47:01.038137 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:01.037806    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:47:01.038137 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:01.037933    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:47:01.256310 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:01.256272    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b" event={"ID":"5ad492c5-f51e-4656-83ee-ab9b28add71c","Type":"ContainerStarted","Data":"030f3a9f1d9e04b32520f033e2a0a752f4528d5dd94b3a01eb9b4d535f72e3fa"}
Jul 29 15:47:01.259774 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:01.259748    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf" event={"ID":"be69d266-eb3a-43bd-b8be-cceea4eda917","Type":"ContainerStarted","Data":"8731eae47f8cd9f512f80ff6c56ae717a21e055414df610385c43ea015ef4af7"}
Jul 29 15:47:01.286421 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:01.286374    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6q7b" podStartSLOduration=4.290724433 podStartE2EDuration="25.286360564s" totalImagesPullingTime="20.995636131s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:46:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:47:01.286192726 +0000 UTC m=+25.728398993" watchObservedRunningTime="2026-07-29 15:47:01.286360564 +0000 UTC m=+25.728566823"
Jul 29 15:47:01.963440 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:01.963393    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rghwf"
Jul 29 15:47:01.964112 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:01.964087    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rghwf"
Jul 29 15:47:02.036875 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:02.036842    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:47:02.037073 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:02.037003    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:47:02.262319 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:02.262241    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rghwf"
Jul 29 15:47:02.262896 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:02.262765    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rghwf"
Jul 29 15:47:03.037591 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:03.037556    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:47:03.037812 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:03.037556    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:47:03.037812 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:03.037675    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:47:03.037812 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:03.037761    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:47:04.040250 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:04.040084    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:47:04.040775 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:04.040331    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:47:04.271645 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:04.271569    2560 generic.go:356] "Generic (PLEG): container finished" podID="eeeada33-be3e-45ce-b580-86be8ee80c48" containerID="ca9dc7be4e85430e3411304e5ff43191d091ba6e1a813075c98650db5520efe4" exitCode=0
Jul 29 15:47:04.271822 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:04.271648    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pbksq" event={"ID":"eeeada33-be3e-45ce-b580-86be8ee80c48","Type":"ContainerDied","Data":"ca9dc7be4e85430e3411304e5ff43191d091ba6e1a813075c98650db5520efe4"}
Jul 29 15:47:04.275192 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:04.275171    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf" event={"ID":"be69d266-eb3a-43bd-b8be-cceea4eda917","Type":"ContainerStarted","Data":"9915e815abe2dc01fcb78f39bcbe1ef394c65e4b7d9fa93ee9fd90e9e84c243b"}
Jul 29 15:47:04.275408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:04.275390    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:47:04.290255 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:04.290239    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:47:04.332292 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:04.332230    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf" podStartSLOduration=9.972327253 podStartE2EDuration="28.332216043s" totalImagesPullingTime="18.35988879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:47:04.330491833 +0000 UTC m=+28.772698091" watchObservedRunningTime="2026-07-29 15:47:04.332216043 +0000 UTC m=+28.774422302"
Jul 29 15:47:04.465644 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:04.465607    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rghwf_78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1/konnectivity-agent/0.log" limit=1024
Jul 29 15:47:04.550404 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:04.550303    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rghwf_78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1/konnectivity-agent/0.log" limit=1024
Jul 29 15:47:05.037292 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:05.037167    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:47:05.037292 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:05.037179    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:47:05.037497 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:05.037296    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:47:05.037778 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:05.037746    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:47:05.253102 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:05.253073    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6w8mg"]
Jul 29 15:47:05.256362 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:05.256334    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-grvsv"]
Jul 29 15:47:05.256592 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:05.256441    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:47:05.256592 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:05.256538    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:47:05.256990 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:05.256970    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z6jwx"]
Jul 29 15:47:05.277736 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:05.277669    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:47:05.277840 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:05.277669    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:47:05.277840 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:05.277780    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:47:05.278457 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:05.278373    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:47:05.278457 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:05.278383    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:47:05.278585 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:05.278469    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:47:05.294696 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:05.294677    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:47:05.564490 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:05.564289    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rghwf_78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1/konnectivity-agent/0.log" limit=1024
Jul 29 15:47:06.281818 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:06.281786    2560 generic.go:356] "Generic (PLEG): container finished" podID="eeeada33-be3e-45ce-b580-86be8ee80c48" containerID="278f2ec6afbba9b74b35a7d3f176a05095bd98ec8b43ebcebfcb1a3c31ae21bf" exitCode=0
Jul 29 15:47:06.282191 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:06.281875    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pbksq" event={"ID":"eeeada33-be3e-45ce-b580-86be8ee80c48","Type":"ContainerDied","Data":"278f2ec6afbba9b74b35a7d3f176a05095bd98ec8b43ebcebfcb1a3c31ae21bf"}
Jul 29 15:47:07.037387 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:07.037357    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:47:07.037387 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:07.037369    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:47:07.037387 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:07.037357    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:47:07.037618 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:07.037459    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:47:07.037618 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:07.037517    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:47:07.037618 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:07.037580    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:47:08.287535 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:08.287500    2560 generic.go:356] "Generic (PLEG): container finished" podID="eeeada33-be3e-45ce-b580-86be8ee80c48" containerID="b7dcd3f9b0e8ac1ddbece73a3bab8deca1ee0cf6f15662fe7ea18d042f6bbe59" exitCode=0
Jul 29 15:47:08.288145 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:08.287569    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pbksq" event={"ID":"eeeada33-be3e-45ce-b580-86be8ee80c48","Type":"ContainerDied","Data":"b7dcd3f9b0e8ac1ddbece73a3bab8deca1ee0cf6f15662fe7ea18d042f6bbe59"}
Jul 29 15:47:09.037582 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:09.037547    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:47:09.037748 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:09.037582    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:47:09.037748 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:09.037597    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:47:09.037748 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:09.037713    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grvsv" podUID="902d7a09-dd0e-466d-95e3-a80d69c1dd11"
Jul 29 15:47:09.037893 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:09.037842    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6jwx" podUID="c45ef580-96eb-4119-89d3-d4dd468e7917"
Jul 29 15:47:09.037954 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:09.037942    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w8mg" podUID="fce7281a-8d9e-4cda-a8ba-7480f20da3ef"
Jul 29 15:47:09.707085 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:09.707051    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs\") pod \"network-metrics-daemon-grvsv\" (UID: \"902d7a09-dd0e-466d-95e3-a80d69c1dd11\") " pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:47:09.707505 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:09.707107    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xclg8\" (UniqueName: \"kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8\") pod \"network-check-target-z6jwx\" (UID: \"c45ef580-96eb-4119-89d3-d4dd468e7917\") " pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:47:09.707505 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:09.707139    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:47:09.707505 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:09.707203    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:47:09.707505 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:09.707245    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:47:09.707505 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:09.707276    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs podName:902d7a09-dd0e-466d-95e3-a80d69c1dd11 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:41.707254984 +0000 UTC m=+66.149461224 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs") pod "network-metrics-daemon-grvsv" (UID: "902d7a09-dd0e-466d-95e3-a80d69c1dd11") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:47:09.707505 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:09.707283    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:47:09.707505 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:09.707292    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret podName:fce7281a-8d9e-4cda-a8ba-7480f20da3ef nodeName:}" failed. No retries permitted until 2026-07-29 15:47:41.70728562 +0000 UTC m=+66.149491857 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret") pod "global-pull-secret-syncer-6w8mg" (UID: "fce7281a-8d9e-4cda-a8ba-7480f20da3ef") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:47:09.707505 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:09.707300    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:47:09.707505 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:09.707314    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xclg8 for pod openshift-network-diagnostics/network-check-target-z6jwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:47:09.707505 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:09.707360    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8 podName:c45ef580-96eb-4119-89d3-d4dd468e7917 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:41.707343723 +0000 UTC m=+66.149549961 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xclg8" (UniqueName: "kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8") pod "network-check-target-z6jwx" (UID: "c45ef580-96eb-4119-89d3-d4dd468e7917") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:47:10.359193 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.359152    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-118.ec2.internal" event="NodeReady"
Jul 29 15:47:10.359381 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.359317    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 15:47:10.397694 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.397660    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78989ff544-62xld"]
Jul 29 15:47:10.399632 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.399610    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.403058 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.403038    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 15:47:10.403346 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.403323    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 15:47:10.403437 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.403373    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 15:47:10.405465 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.405264    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pgnpm\""
Jul 29 15:47:10.409748 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.409722    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mszm6"]
Jul 29 15:47:10.413535 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.413514    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:10.413996 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.413975    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 15:47:10.414512 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.414440    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6wtqz"]
Jul 29 15:47:10.417062 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.416770    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 15:47:10.417062 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.416842    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 15:47:10.417062 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.416860    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lx57w\""
Jul 29 15:47:10.417062 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.416870    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6wtqz"
Jul 29 15:47:10.417252 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.417170    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78989ff544-62xld"]
Jul 29 15:47:10.420581 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.420437    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 15:47:10.420679 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.420580    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 15:47:10.420679 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.420653    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 15:47:10.420779 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.420772    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-h46bm\""
Jul 29 15:47:10.424310 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.424281    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mszm6"]
Jul 29 15:47:10.428030 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.428012    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6wtqz"]
Jul 29 15:47:10.514598 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.514558    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8aac57b2-7e6a-45ec-8d29-96224fcfc997-config-volume\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:10.514598 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.514592    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-image-registry-private-configuration\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.514835 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.514620    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aac57b2-7e6a-45ec-8d29-96224fcfc997-tmp-dir\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:10.514835 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.514690    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.514835 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.514712    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-bound-sa-token\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.514835 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.514728    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r7g4\" (UniqueName: \"kubernetes.io/projected/4f7e28c6-0d23-445a-9b23-725d4dc65be1-kube-api-access-7r7g4\") pod \"ingress-canary-6wtqz\" (UID: \"4f7e28c6-0d23-445a-9b23-725d4dc65be1\") " pod="openshift-ingress-canary/ingress-canary-6wtqz"
Jul 29 15:47:10.514835 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.514745    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:10.514835 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.514779    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crr8l\" (UniqueName: \"kubernetes.io/projected/8aac57b2-7e6a-45ec-8d29-96224fcfc997-kube-api-access-crr8l\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:10.515205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.514858    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-installation-pull-secrets\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.515205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.514909    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-trusted-ca\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.515205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.514953    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6kc7\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-kube-api-access-m6kc7\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.515205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.514990    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-ca-trust-extracted\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.515205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.515013    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-certificates\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.515205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.515036    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f7e28c6-0d23-445a-9b23-725d4dc65be1-cert\") pod \"ingress-canary-6wtqz\" (UID: \"4f7e28c6-0d23-445a-9b23-725d4dc65be1\") " pod="openshift-ingress-canary/ingress-canary-6wtqz"
Jul 29 15:47:10.616335 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616260    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-installation-pull-secrets\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.616335 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616297    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-trusted-ca\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.616335 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616328    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6kc7\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-kube-api-access-m6kc7\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.616617 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616362    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-ca-trust-extracted\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.616617 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616382    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-certificates\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.616617 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616441    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f7e28c6-0d23-445a-9b23-725d4dc65be1-cert\") pod \"ingress-canary-6wtqz\" (UID: \"4f7e28c6-0d23-445a-9b23-725d4dc65be1\") " pod="openshift-ingress-canary/ingress-canary-6wtqz"
Jul 29 15:47:10.616617 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616511    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8aac57b2-7e6a-45ec-8d29-96224fcfc997-config-volume\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:10.616617 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616545    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-image-registry-private-configuration\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.616617 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616570    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aac57b2-7e6a-45ec-8d29-96224fcfc997-tmp-dir\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:10.616617 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616610    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.616986 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616635    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-bound-sa-token\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.616986 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616656    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r7g4\" (UniqueName: \"kubernetes.io/projected/4f7e28c6-0d23-445a-9b23-725d4dc65be1-kube-api-access-7r7g4\") pod \"ingress-canary-6wtqz\" (UID: \"4f7e28c6-0d23-445a-9b23-725d4dc65be1\") " pod="openshift-ingress-canary/ingress-canary-6wtqz"
Jul 29 15:47:10.616986 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616680    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:10.616986 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616708    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crr8l\" (UniqueName: \"kubernetes.io/projected/8aac57b2-7e6a-45ec-8d29-96224fcfc997-kube-api-access-crr8l\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:10.616986 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.616814    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-ca-trust-extracted\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.617300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.617188    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aac57b2-7e6a-45ec-8d29-96224fcfc997-tmp-dir\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:10.617300 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:10.617210    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:47:10.617300 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:10.617260    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls podName:8aac57b2-7e6a-45ec-8d29-96224fcfc997 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:11.11724471 +0000 UTC m=+35.559450963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls") pod "dns-default-mszm6" (UID: "8aac57b2-7e6a-45ec-8d29-96224fcfc997") : secret "dns-default-metrics-tls" not found
Jul 29 15:47:10.617300 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:10.617276    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:47:10.617300 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:10.617292    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78989ff544-62xld: secret "image-registry-tls" not found
Jul 29 15:47:10.617564 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:10.617345    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls podName:5dd620e8-a4e4-45e4-be96-0bc62db9a87b nodeName:}" failed. No retries permitted until 2026-07-29 15:47:11.11732908 +0000 UTC m=+35.559535324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls") pod "image-registry-78989ff544-62xld" (UID: "5dd620e8-a4e4-45e4-be96-0bc62db9a87b") : secret "image-registry-tls" not found
Jul 29 15:47:10.617564 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.617367    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-trusted-ca\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.617798 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.617777    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8aac57b2-7e6a-45ec-8d29-96224fcfc997-config-volume\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:10.617870 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.617801    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-certificates\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.621065 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.621043    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-installation-pull-secrets\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.621158 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.621066    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f7e28c6-0d23-445a-9b23-725d4dc65be1-cert\") pod \"ingress-canary-6wtqz\" (UID: \"4f7e28c6-0d23-445a-9b23-725d4dc65be1\") " pod="openshift-ingress-canary/ingress-canary-6wtqz"
Jul 29 15:47:10.621158 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.621043    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-image-registry-private-configuration\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.626455 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.626417    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r7g4\" (UniqueName: \"kubernetes.io/projected/4f7e28c6-0d23-445a-9b23-725d4dc65be1-kube-api-access-7r7g4\") pod \"ingress-canary-6wtqz\" (UID: \"4f7e28c6-0d23-445a-9b23-725d4dc65be1\") " pod="openshift-ingress-canary/ingress-canary-6wtqz"
Jul 29 15:47:10.627195 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.627175    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-bound-sa-token\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.627417 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.627393    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6kc7\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-kube-api-access-m6kc7\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:10.627486 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.627467    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crr8l\" (UniqueName: \"kubernetes.io/projected/8aac57b2-7e6a-45ec-8d29-96224fcfc997-kube-api-access-crr8l\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:10.732828 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.732796    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6wtqz"
Jul 29 15:47:10.876316 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:10.876104    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6wtqz"]
Jul 29 15:47:11.037344 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:11.037306    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:47:11.037516 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:11.037308    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:47:11.037516 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:11.037458    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:47:11.040339 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:11.040314    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-28zhr\""
Jul 29 15:47:11.040475 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:11.040347    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hf4f8\""
Jul 29 15:47:11.040475 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:11.040458    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 15:47:11.040591 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:11.040458    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 15:47:11.040692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:11.040672    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 15:47:11.041023 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:11.041004    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 15:47:11.121247 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:11.121216    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:11.121247 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:11.121258    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:11.121479 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:11.121379    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:47:11.121479 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:11.121437    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls podName:8aac57b2-7e6a-45ec-8d29-96224fcfc997 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:12.121419109 +0000 UTC m=+36.563625348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls") pod "dns-default-mszm6" (UID: "8aac57b2-7e6a-45ec-8d29-96224fcfc997") : secret "dns-default-metrics-tls" not found
Jul 29 15:47:11.121479 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:11.121383    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:47:11.121479 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:11.121472    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78989ff544-62xld: secret "image-registry-tls" not found
Jul 29 15:47:11.121662 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:11.121510    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls podName:5dd620e8-a4e4-45e4-be96-0bc62db9a87b nodeName:}" failed. No retries permitted until 2026-07-29 15:47:12.121498617 +0000 UTC m=+36.563704859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls") pod "image-registry-78989ff544-62xld" (UID: "5dd620e8-a4e4-45e4-be96-0bc62db9a87b") : secret "image-registry-tls" not found
Jul 29 15:47:12.128495 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:12.128462    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:12.128951 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:12.128591    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:12.128951 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:12.128647    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:47:12.128951 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:12.128703    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:47:12.128951 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:12.128719    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78989ff544-62xld: secret "image-registry-tls" not found
Jul 29 15:47:12.128951 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:12.128727    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls podName:8aac57b2-7e6a-45ec-8d29-96224fcfc997 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:14.128704813 +0000 UTC m=+38.570911067 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls") pod "dns-default-mszm6" (UID: "8aac57b2-7e6a-45ec-8d29-96224fcfc997") : secret "dns-default-metrics-tls" not found
Jul 29 15:47:12.128951 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:12.128767    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls podName:5dd620e8-a4e4-45e4-be96-0bc62db9a87b nodeName:}" failed. No retries permitted until 2026-07-29 15:47:14.128750339 +0000 UTC m=+38.570956587 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls") pod "image-registry-78989ff544-62xld" (UID: "5dd620e8-a4e4-45e4-be96-0bc62db9a87b") : secret "image-registry-tls" not found
Jul 29 15:47:12.322105 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:47:12.322071    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f7e28c6_0d23_445a_9b23_725d4dc65be1.slice/crio-59cc9530182217bd06d469d27990a230b63a3a1c0350ead0cbb69b96f4b7db47 WatchSource:0}: Error finding container 59cc9530182217bd06d469d27990a230b63a3a1c0350ead0cbb69b96f4b7db47: Status 404 returned error can't find the container with id 59cc9530182217bd06d469d27990a230b63a3a1c0350ead0cbb69b96f4b7db47
Jul 29 15:47:13.302055 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:13.302011    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6wtqz" event={"ID":"4f7e28c6-0d23-445a-9b23-725d4dc65be1","Type":"ContainerStarted","Data":"59cc9530182217bd06d469d27990a230b63a3a1c0350ead0cbb69b96f4b7db47"}
Jul 29 15:47:13.305293 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:13.305264    2560 generic.go:356] "Generic (PLEG): container finished" podID="eeeada33-be3e-45ce-b580-86be8ee80c48" containerID="c6e0bb63f06bb63b2fd6725c6690021d6356478245f92da51bddf65c2316774b" exitCode=0
Jul 29 15:47:13.305422 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:13.305311    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pbksq" event={"ID":"eeeada33-be3e-45ce-b580-86be8ee80c48","Type":"ContainerDied","Data":"c6e0bb63f06bb63b2fd6725c6690021d6356478245f92da51bddf65c2316774b"}
Jul 29 15:47:14.140804 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:14.140764    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:14.140804 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:14.140808    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:14.141175 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:14.140948    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:47:14.141175 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:14.140961    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:47:14.141175 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:14.140965    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78989ff544-62xld: secret "image-registry-tls" not found
Jul 29 15:47:14.141175 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:14.141028    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls podName:5dd620e8-a4e4-45e4-be96-0bc62db9a87b nodeName:}" failed. No retries permitted until 2026-07-29 15:47:18.141008268 +0000 UTC m=+42.583214508 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls") pod "image-registry-78989ff544-62xld" (UID: "5dd620e8-a4e4-45e4-be96-0bc62db9a87b") : secret "image-registry-tls" not found
Jul 29 15:47:14.141175 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:14.141050    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls podName:8aac57b2-7e6a-45ec-8d29-96224fcfc997 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:18.141040579 +0000 UTC m=+42.583246820 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls") pod "dns-default-mszm6" (UID: "8aac57b2-7e6a-45ec-8d29-96224fcfc997") : secret "dns-default-metrics-tls" not found
Jul 29 15:47:14.311437 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:14.311245    2560 generic.go:356] "Generic (PLEG): container finished" podID="eeeada33-be3e-45ce-b580-86be8ee80c48" containerID="ac72e6a9223b26a2aa5aa6f3a7f9201f261183ba5556e66aecf8b681fdad1cc5" exitCode=0
Jul 29 15:47:14.311827 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:14.311334    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pbksq" event={"ID":"eeeada33-be3e-45ce-b580-86be8ee80c48","Type":"ContainerDied","Data":"ac72e6a9223b26a2aa5aa6f3a7f9201f261183ba5556e66aecf8b681fdad1cc5"}
Jul 29 15:47:15.317621 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:15.317550    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pbksq" event={"ID":"eeeada33-be3e-45ce-b580-86be8ee80c48","Type":"ContainerStarted","Data":"ddc58088d5cbd98ca2f39ef17538bd991ced5c8c5c978c5bde2df03758e72822"}
Jul 29 15:47:15.347530 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:15.347409    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pbksq" podStartSLOduration=9.317596728 podStartE2EDuration="39.347391465s" totalImagesPullingTime="30.029794737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:46:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:47:15.345606767 +0000 UTC m=+39.787813026" watchObservedRunningTime="2026-07-29 15:47:15.347391465 +0000 UTC m=+39.789597725"
Jul 29 15:47:16.323701 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:16.323662    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6wtqz" event={"ID":"4f7e28c6-0d23-445a-9b23-725d4dc65be1","Type":"ContainerStarted","Data":"40ca85e104099550c402aa580cf8e64621dcbd134e63cb15c57b0021e5b19d69"}
Jul 29 15:47:16.339312 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:16.339255    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6wtqz" podStartSLOduration=3.339289245 podStartE2EDuration="6.339237709s" totalImagesPullingTime="2.999948464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:47:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:47:16.339078643 +0000 UTC m=+40.781284903" watchObservedRunningTime="2026-07-29 15:47:16.339237709 +0000 UTC m=+40.781443970"
Jul 29 15:47:18.173687 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.173650    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:18.174177 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.173760    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:18.174177 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:18.173807    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:47:18.174177 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:18.173886    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls podName:8aac57b2-7e6a-45ec-8d29-96224fcfc997 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:26.173859229 +0000 UTC m=+50.616065469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls") pod "dns-default-mszm6" (UID: "8aac57b2-7e6a-45ec-8d29-96224fcfc997") : secret "dns-default-metrics-tls" not found
Jul 29 15:47:18.174177 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:18.173887    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:47:18.174177 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:18.173904    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78989ff544-62xld: secret "image-registry-tls" not found
Jul 29 15:47:18.174177 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:18.173976    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls podName:5dd620e8-a4e4-45e4-be96-0bc62db9a87b nodeName:}" failed. No retries permitted until 2026-07-29 15:47:26.173960494 +0000 UTC m=+50.616166731 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls") pod "image-registry-78989ff544-62xld" (UID: "5dd620e8-a4e4-45e4-be96-0bc62db9a87b") : secret "image-registry-tls" not found
Jul 29 15:47:18.765926 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.765893    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2k9gn"]
Jul 29 15:47:18.798097 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.798069    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2k9gn"]
Jul 29 15:47:18.798217 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.798167    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:18.802769 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.802747    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sv279\""
Jul 29 15:47:18.803967 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.803947    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 15:47:18.804071 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.803977    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 15:47:18.806042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.806025    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 15:47:18.811840 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.811826    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 15:47:18.978929 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.978886    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr9sn\" (UniqueName: \"kubernetes.io/projected/41f96031-45cd-4059-9d82-e277dd98bb71-kube-api-access-sr9sn\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:18.979096 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.978965    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f96031-45cd-4059-9d82-e277dd98bb71-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:18.979096 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.979005    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/41f96031-45cd-4059-9d82-e277dd98bb71-crio-socket\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:18.979096 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.979084    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/41f96031-45cd-4059-9d82-e277dd98bb71-data-volume\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:18.979195 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:18.979113    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:19.080457 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:19.080369    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr9sn\" (UniqueName: \"kubernetes.io/projected/41f96031-45cd-4059-9d82-e277dd98bb71-kube-api-access-sr9sn\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:19.080457 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:19.080423    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f96031-45cd-4059-9d82-e277dd98bb71-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:19.080680 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:19.080469    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/41f96031-45cd-4059-9d82-e277dd98bb71-crio-socket\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:19.080680 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:19.080527    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/41f96031-45cd-4059-9d82-e277dd98bb71-data-volume\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:19.080680 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:19.080549    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:19.080680 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:19.080559    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 15:47:19.080680 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:19.080631    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41f96031-45cd-4059-9d82-e277dd98bb71-kube-rbac-proxy-cm podName:41f96031-45cd-4059-9d82-e277dd98bb71 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:19.580610544 +0000 UTC m=+44.022816784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/41f96031-45cd-4059-9d82-e277dd98bb71-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2k9gn" (UID: "41f96031-45cd-4059-9d82-e277dd98bb71") : configmap "kube-rbac-proxy" not found
Jul 29 15:47:19.080680 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:19.080661    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:47:19.080893 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:19.080722    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls podName:41f96031-45cd-4059-9d82-e277dd98bb71 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:19.580706071 +0000 UTC m=+44.022912308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2k9gn" (UID: "41f96031-45cd-4059-9d82-e277dd98bb71") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:47:19.080893 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:19.080745    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/41f96031-45cd-4059-9d82-e277dd98bb71-crio-socket\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:19.080893 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:19.080881    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/41f96031-45cd-4059-9d82-e277dd98bb71-data-volume\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:19.096988 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:19.096962    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr9sn\" (UniqueName: \"kubernetes.io/projected/41f96031-45cd-4059-9d82-e277dd98bb71-kube-api-access-sr9sn\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:19.583575 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:19.583540    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:19.584007 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:19.583601    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f96031-45cd-4059-9d82-e277dd98bb71-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:19.584007 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:19.583659    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 15:47:19.584007 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:19.583709    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41f96031-45cd-4059-9d82-e277dd98bb71-kube-rbac-proxy-cm podName:41f96031-45cd-4059-9d82-e277dd98bb71 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:20.583694725 +0000 UTC m=+45.025900971 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/41f96031-45cd-4059-9d82-e277dd98bb71-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2k9gn" (UID: "41f96031-45cd-4059-9d82-e277dd98bb71") : configmap "kube-rbac-proxy" not found
Jul 29 15:47:19.584007 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:19.583708    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:47:19.584007 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:19.583788    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls podName:41f96031-45cd-4059-9d82-e277dd98bb71 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:20.58376981 +0000 UTC m=+45.025976052 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2k9gn" (UID: "41f96031-45cd-4059-9d82-e277dd98bb71") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:47:20.590805 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.590771    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:20.591230 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.590837    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f96031-45cd-4059-9d82-e277dd98bb71-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:20.591230 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:20.590942    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 15:47:20.591230 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:20.590992    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41f96031-45cd-4059-9d82-e277dd98bb71-kube-rbac-proxy-cm podName:41f96031-45cd-4059-9d82-e277dd98bb71 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:22.590978263 +0000 UTC m=+47.033184500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/41f96031-45cd-4059-9d82-e277dd98bb71-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2k9gn" (UID: "41f96031-45cd-4059-9d82-e277dd98bb71") : configmap "kube-rbac-proxy" not found
Jul 29 15:47:20.591230 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:20.590939    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:47:20.591230 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:20.591094    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls podName:41f96031-45cd-4059-9d82-e277dd98bb71 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:22.591078842 +0000 UTC m=+47.033285079 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2k9gn" (UID: "41f96031-45cd-4059-9d82-e277dd98bb71") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:47:20.687741 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.687709    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-mfp4c"]
Jul 29 15:47:20.699173 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.699150    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-mfp4c"
Jul 29 15:47:20.703637 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.703615    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-mfp4c"]
Jul 29 15:47:20.704071 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.704054    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 29 15:47:20.704381 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.704366    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4b5fs\""
Jul 29 15:47:20.704475 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.704393    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 29 15:47:20.704475 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.704393    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 29 15:47:20.704621 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.704607    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 29 15:47:20.773641 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.773616    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kgn8d_6fde166c-e320-4357-8289-4450abb2cc06/dns-node-resolver/0.log"
Jul 29 15:47:20.893998 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.893969    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/17f961b8-692f-4852-a27c-08486e2d09ee-signing-key\") pod \"service-ca-658d7b7c78-mfp4c\" (UID: \"17f961b8-692f-4852-a27c-08486e2d09ee\") " pod="openshift-service-ca/service-ca-658d7b7c78-mfp4c"
Jul 29 15:47:20.894173 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.894012    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/17f961b8-692f-4852-a27c-08486e2d09ee-signing-cabundle\") pod \"service-ca-658d7b7c78-mfp4c\" (UID: \"17f961b8-692f-4852-a27c-08486e2d09ee\") " pod="openshift-service-ca/service-ca-658d7b7c78-mfp4c"
Jul 29 15:47:20.894173 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.894040    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdzd2\" (UniqueName: \"kubernetes.io/projected/17f961b8-692f-4852-a27c-08486e2d09ee-kube-api-access-kdzd2\") pod \"service-ca-658d7b7c78-mfp4c\" (UID: \"17f961b8-692f-4852-a27c-08486e2d09ee\") " pod="openshift-service-ca/service-ca-658d7b7c78-mfp4c"
Jul 29 15:47:20.995319 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.995292    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/17f961b8-692f-4852-a27c-08486e2d09ee-signing-key\") pod \"service-ca-658d7b7c78-mfp4c\" (UID: \"17f961b8-692f-4852-a27c-08486e2d09ee\") " pod="openshift-service-ca/service-ca-658d7b7c78-mfp4c"
Jul 29 15:47:20.995471 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.995325    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/17f961b8-692f-4852-a27c-08486e2d09ee-signing-cabundle\") pod \"service-ca-658d7b7c78-mfp4c\" (UID: \"17f961b8-692f-4852-a27c-08486e2d09ee\") " pod="openshift-service-ca/service-ca-658d7b7c78-mfp4c"
Jul 29 15:47:20.995471 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.995349    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdzd2\" (UniqueName: \"kubernetes.io/projected/17f961b8-692f-4852-a27c-08486e2d09ee-kube-api-access-kdzd2\") pod \"service-ca-658d7b7c78-mfp4c\" (UID: \"17f961b8-692f-4852-a27c-08486e2d09ee\") " pod="openshift-service-ca/service-ca-658d7b7c78-mfp4c"
Jul 29 15:47:20.995989 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.995964    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/17f961b8-692f-4852-a27c-08486e2d09ee-signing-cabundle\") pod \"service-ca-658d7b7c78-mfp4c\" (UID: \"17f961b8-692f-4852-a27c-08486e2d09ee\") " pod="openshift-service-ca/service-ca-658d7b7c78-mfp4c"
Jul 29 15:47:20.997783 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:20.997759    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/17f961b8-692f-4852-a27c-08486e2d09ee-signing-key\") pod \"service-ca-658d7b7c78-mfp4c\" (UID: \"17f961b8-692f-4852-a27c-08486e2d09ee\") " pod="openshift-service-ca/service-ca-658d7b7c78-mfp4c"
Jul 29 15:47:21.007407 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:21.007382    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdzd2\" (UniqueName: \"kubernetes.io/projected/17f961b8-692f-4852-a27c-08486e2d09ee-kube-api-access-kdzd2\") pod \"service-ca-658d7b7c78-mfp4c\" (UID: \"17f961b8-692f-4852-a27c-08486e2d09ee\") " pod="openshift-service-ca/service-ca-658d7b7c78-mfp4c"
Jul 29 15:47:21.008991 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:21.008968    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-mfp4c"
Jul 29 15:47:21.161757 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:21.161666    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-mfp4c"]
Jul 29 15:47:21.165505 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:47:21.165477    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17f961b8_692f_4852_a27c_08486e2d09ee.slice/crio-370b67e3a5f247a4c7e73a11f9c903e84fb83d0b00899b407d1b812878a4e100 WatchSource:0}: Error finding container 370b67e3a5f247a4c7e73a11f9c903e84fb83d0b00899b407d1b812878a4e100: Status 404 returned error can't find the container with id 370b67e3a5f247a4c7e73a11f9c903e84fb83d0b00899b407d1b812878a4e100
Jul 29 15:47:21.336411 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:21.336378    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-mfp4c" event={"ID":"17f961b8-692f-4852-a27c-08486e2d09ee","Type":"ContainerStarted","Data":"370b67e3a5f247a4c7e73a11f9c903e84fb83d0b00899b407d1b812878a4e100"}
Jul 29 15:47:21.576677 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:21.576603    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5l247_5f5c78df-f246-43b5-a4c2-7704c6af4c1d/node-ca/0.log"
Jul 29 15:47:22.374792 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:22.374748    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6wtqz_4f7e28c6-0d23-445a-9b23-725d4dc65be1/serve-healthcheck-canary/0.log"
Jul 29 15:47:22.609447 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:22.609404    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f96031-45cd-4059-9d82-e277dd98bb71-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:22.609614 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:22.609514    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:22.609669 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:22.609646    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:47:22.609724 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:22.609711    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls podName:41f96031-45cd-4059-9d82-e277dd98bb71 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:26.609693225 +0000 UTC m=+51.051899463 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2k9gn" (UID: "41f96031-45cd-4059-9d82-e277dd98bb71") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:47:22.610001 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:22.609981    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/41f96031-45cd-4059-9d82-e277dd98bb71-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:24.345796 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:24.345764    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-mfp4c" event={"ID":"17f961b8-692f-4852-a27c-08486e2d09ee","Type":"ContainerStarted","Data":"470cc611060cd01c87763e32cfd109d54612cbf4a5b120aad466757eed790dcd"}
Jul 29 15:47:24.391656 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:24.391605    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-mfp4c" podStartSLOduration=2.245780221 podStartE2EDuration="4.391590263s" totalImagesPullingTime="2.145810042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:47:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:47:24.391060997 +0000 UTC m=+48.833267261" watchObservedRunningTime="2026-07-29 15:47:24.391590263 +0000 UTC m=+48.833796557"
Jul 29 15:47:26.238112 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:26.238081    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:26.238569 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:26.238129    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:26.238569 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:26.238248    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:47:26.238569 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:26.238270    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78989ff544-62xld: secret "image-registry-tls" not found
Jul 29 15:47:26.238569 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:26.238298    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:47:26.238569 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:26.238325    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls podName:5dd620e8-a4e4-45e4-be96-0bc62db9a87b nodeName:}" failed. No retries permitted until 2026-07-29 15:47:42.238308963 +0000 UTC m=+66.680515203 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls") pod "image-registry-78989ff544-62xld" (UID: "5dd620e8-a4e4-45e4-be96-0bc62db9a87b") : secret "image-registry-tls" not found
Jul 29 15:47:26.238569 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:26.238384    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls podName:8aac57b2-7e6a-45ec-8d29-96224fcfc997 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:42.238367879 +0000 UTC m=+66.680574136 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls") pod "dns-default-mszm6" (UID: "8aac57b2-7e6a-45ec-8d29-96224fcfc997") : secret "dns-default-metrics-tls" not found
Jul 29 15:47:26.641972 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:26.641945    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:26.642126 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:26.642107    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:47:26.642216 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:26.642176    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls podName:41f96031-45cd-4059-9d82-e277dd98bb71 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:34.642159145 +0000 UTC m=+59.084365381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2k9gn" (UID: "41f96031-45cd-4059-9d82-e277dd98bb71") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:47:34.711775 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:34.711733    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:34.714029 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:34.714009    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/41f96031-45cd-4059-9d82-e277dd98bb71-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2k9gn\" (UID: \"41f96031-45cd-4059-9d82-e277dd98bb71\") " pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:35.007158 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:35.007071    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2k9gn"
Jul 29 15:47:35.130938 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:35.130882    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2k9gn"]
Jul 29 15:47:35.133957 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:47:35.133932    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41f96031_45cd_4059_9d82_e277dd98bb71.slice/crio-24096ed114b4f53a915ecf180aff060d0fd6baa59987422d050ae2dbf1baf3b7 WatchSource:0}: Error finding container 24096ed114b4f53a915ecf180aff060d0fd6baa59987422d050ae2dbf1baf3b7: Status 404 returned error can't find the container with id 24096ed114b4f53a915ecf180aff060d0fd6baa59987422d050ae2dbf1baf3b7
Jul 29 15:47:35.373781 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:35.373702    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2k9gn" event={"ID":"41f96031-45cd-4059-9d82-e277dd98bb71","Type":"ContainerStarted","Data":"24096ed114b4f53a915ecf180aff060d0fd6baa59987422d050ae2dbf1baf3b7"}
Jul 29 15:47:37.295464 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:37.295430    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bbndf"
Jul 29 15:47:40.387713 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:40.387629    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2k9gn" event={"ID":"41f96031-45cd-4059-9d82-e277dd98bb71","Type":"ContainerStarted","Data":"5d3453b082b11c2fa85974a844c57b997d4fbc246c83e2ed1917befce7d3cd73"}
Jul 29 15:47:41.775383 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.775339    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs\") pod \"network-metrics-daemon-grvsv\" (UID: \"902d7a09-dd0e-466d-95e3-a80d69c1dd11\") " pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:47:41.775843 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.775403    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xclg8\" (UniqueName: \"kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8\") pod \"network-check-target-z6jwx\" (UID: \"c45ef580-96eb-4119-89d3-d4dd468e7917\") " pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:47:41.775843 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.775445    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:47:41.778551 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.778526    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 15:47:41.779838 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.779819    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 15:47:41.779940 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.779836    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 15:47:41.787886 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.787863    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/902d7a09-dd0e-466d-95e3-a80d69c1dd11-metrics-certs\") pod \"network-metrics-daemon-grvsv\" (UID: \"902d7a09-dd0e-466d-95e3-a80d69c1dd11\") " pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:47:41.788315 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.788295    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fce7281a-8d9e-4cda-a8ba-7480f20da3ef-original-pull-secret\") pod \"global-pull-secret-syncer-6w8mg\" (UID: \"fce7281a-8d9e-4cda-a8ba-7480f20da3ef\") " pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:47:41.788733 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.788716    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 15:47:41.798389 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.798367    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xclg8\" (UniqueName: \"kubernetes.io/projected/c45ef580-96eb-4119-89d3-d4dd468e7917-kube-api-access-xclg8\") pod \"network-check-target-z6jwx\" (UID: \"c45ef580-96eb-4119-89d3-d4dd468e7917\") " pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:47:41.952704 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.952664    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-28zhr\""
Jul 29 15:47:41.955325 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.955303    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w8mg"
Jul 29 15:47:41.960456 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.960427    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grvsv"
Jul 29 15:47:41.964095 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.964074    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hf4f8\""
Jul 29 15:47:41.971330 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:41.971311    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:47:42.279583 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:42.279543    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:42.279763 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:42.279592    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:42.282695 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:42.282667    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aac57b2-7e6a-45ec-8d29-96224fcfc997-metrics-tls\") pod \"dns-default-mszm6\" (UID: \"8aac57b2-7e6a-45ec-8d29-96224fcfc997\") " pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:42.282878 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:42.282858    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls\") pod \"image-registry-78989ff544-62xld\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") " pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:42.514524 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:42.514496    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pgnpm\""
Jul 29 15:47:42.521984 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:42.521957    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:42.529146 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:42.529122    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lx57w\""
Jul 29 15:47:42.536620 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:42.536559    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:44.140540 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:44.140484    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mszm6"]
Jul 29 15:47:44.144746 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:47:44.144693    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aac57b2_7e6a_45ec_8d29_96224fcfc997.slice/crio-d6261f993734033d18b3fde3d246ac4467bb9662fe88145f23ee1301a8764b24 WatchSource:0}: Error finding container d6261f993734033d18b3fde3d246ac4467bb9662fe88145f23ee1301a8764b24: Status 404 returned error can't find the container with id d6261f993734033d18b3fde3d246ac4467bb9662fe88145f23ee1301a8764b24
Jul 29 15:47:44.152221 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:44.152201    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z6jwx"]
Jul 29 15:47:44.155398 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:47:44.155374    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc45ef580_96eb_4119_89d3_d4dd468e7917.slice/crio-e6aa32f9b385540ae53d6c943138d6a469dff6dc279abbf06e3b178985741a6e WatchSource:0}: Error finding container e6aa32f9b385540ae53d6c943138d6a469dff6dc279abbf06e3b178985741a6e: Status 404 returned error can't find the container with id e6aa32f9b385540ae53d6c943138d6a469dff6dc279abbf06e3b178985741a6e
Jul 29 15:47:44.374399 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:44.374332    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6w8mg"]
Jul 29 15:47:44.377448 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:44.377416    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-grvsv"]
Jul 29 15:47:44.378329 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:44.378307    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78989ff544-62xld"]
Jul 29 15:47:44.382011 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:47:44.381990    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd620e8_a4e4_45e4_be96_0bc62db9a87b.slice/crio-40613f87cb4a5cc973a46cbb4f5d5253f1391b67d547adc0d4395e9c1a91bb2b WatchSource:0}: Error finding container 40613f87cb4a5cc973a46cbb4f5d5253f1391b67d547adc0d4395e9c1a91bb2b: Status 404 returned error can't find the container with id 40613f87cb4a5cc973a46cbb4f5d5253f1391b67d547adc0d4395e9c1a91bb2b
Jul 29 15:47:44.382790 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:47:44.382770    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod902d7a09_dd0e_466d_95e3_a80d69c1dd11.slice/crio-cc06a1a70701d5ab073daf3fc85c858c523d4586a4921b75dee0a69ea66f4b71 WatchSource:0}: Error finding container cc06a1a70701d5ab073daf3fc85c858c523d4586a4921b75dee0a69ea66f4b71: Status 404 returned error can't find the container with id cc06a1a70701d5ab073daf3fc85c858c523d4586a4921b75dee0a69ea66f4b71
Jul 29 15:47:44.399034 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:44.399011    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6w8mg" event={"ID":"fce7281a-8d9e-4cda-a8ba-7480f20da3ef","Type":"ContainerStarted","Data":"9216f6b962b9f4bc865db35b1dedadacfac418abe2fcac36c782ab622e30b043"}
Jul 29 15:47:44.400095 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:44.400075    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mszm6" event={"ID":"8aac57b2-7e6a-45ec-8d29-96224fcfc997","Type":"ContainerStarted","Data":"d6261f993734033d18b3fde3d246ac4467bb9662fe88145f23ee1301a8764b24"}
Jul 29 15:47:44.401112 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:44.401088    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z6jwx" event={"ID":"c45ef580-96eb-4119-89d3-d4dd468e7917","Type":"ContainerStarted","Data":"e6aa32f9b385540ae53d6c943138d6a469dff6dc279abbf06e3b178985741a6e"}
Jul 29 15:47:44.402192 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:44.402168    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-grvsv" event={"ID":"902d7a09-dd0e-466d-95e3-a80d69c1dd11","Type":"ContainerStarted","Data":"cc06a1a70701d5ab073daf3fc85c858c523d4586a4921b75dee0a69ea66f4b71"}
Jul 29 15:47:44.403615 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:44.403598    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2k9gn" event={"ID":"41f96031-45cd-4059-9d82-e277dd98bb71","Type":"ContainerStarted","Data":"ebd074694b2c0836d9ce5465c0d27e9194374fdb85ade952c983c7e0d0ac8818"}
Jul 29 15:47:44.404501 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:44.404485    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78989ff544-62xld" event={"ID":"5dd620e8-a4e4-45e4-be96-0bc62db9a87b","Type":"ContainerStarted","Data":"40613f87cb4a5cc973a46cbb4f5d5253f1391b67d547adc0d4395e9c1a91bb2b"}
Jul 29 15:47:45.140864 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.139789    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"]
Jul 29 15:47:45.142405 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.142382    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"
Jul 29 15:47:45.152781 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.152131    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 29 15:47:45.153529 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.153509    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 29 15:47:45.153798 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.153781    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 29 15:47:45.154041 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.154021    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 15:47:45.157300 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.156238    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668845584b-bk28v"]
Jul 29 15:47:45.159497 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.159003    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668845584b-bk28v"
Jul 29 15:47:45.163155 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.163133    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-xggps\""
Jul 29 15:47:45.163771 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.163753    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 29 15:47:45.165174 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.165143    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"]
Jul 29 15:47:45.186273 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.186248    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668845584b-bk28v"]
Jul 29 15:47:45.203029 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.202750    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56q8n\" (UniqueName: \"kubernetes.io/projected/16337e01-6e7f-4222-8e91-ce12fffe034b-kube-api-access-56q8n\") pod \"klusterlet-addon-workmgr-5984b86486-sr5c5\" (UID: \"16337e01-6e7f-4222-8e91-ce12fffe034b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"
Jul 29 15:47:45.203029 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.202880    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/16337e01-6e7f-4222-8e91-ce12fffe034b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5984b86486-sr5c5\" (UID: \"16337e01-6e7f-4222-8e91-ce12fffe034b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"
Jul 29 15:47:45.207340 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.202935    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16337e01-6e7f-4222-8e91-ce12fffe034b-tmp\") pod \"klusterlet-addon-workmgr-5984b86486-sr5c5\" (UID: \"16337e01-6e7f-4222-8e91-ce12fffe034b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"
Jul 29 15:47:45.304065 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.304027    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/16337e01-6e7f-4222-8e91-ce12fffe034b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5984b86486-sr5c5\" (UID: \"16337e01-6e7f-4222-8e91-ce12fffe034b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"
Jul 29 15:47:45.304260 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.304094    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5l6c\" (UniqueName: \"kubernetes.io/projected/3d7a2e79-8998-43d4-ba83-a62030f33866-kube-api-access-h5l6c\") pod \"managed-serviceaccount-addon-agent-668845584b-bk28v\" (UID: \"3d7a2e79-8998-43d4-ba83-a62030f33866\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668845584b-bk28v"
Jul 29 15:47:45.304260 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.304147    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56q8n\" (UniqueName: \"kubernetes.io/projected/16337e01-6e7f-4222-8e91-ce12fffe034b-kube-api-access-56q8n\") pod \"klusterlet-addon-workmgr-5984b86486-sr5c5\" (UID: \"16337e01-6e7f-4222-8e91-ce12fffe034b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"
Jul 29 15:47:45.304260 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.304185    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16337e01-6e7f-4222-8e91-ce12fffe034b-tmp\") pod \"klusterlet-addon-workmgr-5984b86486-sr5c5\" (UID: \"16337e01-6e7f-4222-8e91-ce12fffe034b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"
Jul 29 15:47:45.304260 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.304220    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d7a2e79-8998-43d4-ba83-a62030f33866-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-668845584b-bk28v\" (UID: \"3d7a2e79-8998-43d4-ba83-a62030f33866\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668845584b-bk28v"
Jul 29 15:47:45.305488 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.305278    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16337e01-6e7f-4222-8e91-ce12fffe034b-tmp\") pod \"klusterlet-addon-workmgr-5984b86486-sr5c5\" (UID: \"16337e01-6e7f-4222-8e91-ce12fffe034b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"
Jul 29 15:47:45.311513 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.310947    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-gdtj7"]
Jul 29 15:47:45.311513 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.311441    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/16337e01-6e7f-4222-8e91-ce12fffe034b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5984b86486-sr5c5\" (UID: \"16337e01-6e7f-4222-8e91-ce12fffe034b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"
Jul 29 15:47:45.315848 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.315024    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-gdtj7"
Jul 29 15:47:45.319584 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.319538    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78989ff544-62xld"]
Jul 29 15:47:45.320854 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.320679    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-572vs\""
Jul 29 15:47:45.321955 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.321670    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 15:47:45.333268 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.333245    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56q8n\" (UniqueName: \"kubernetes.io/projected/16337e01-6e7f-4222-8e91-ce12fffe034b-kube-api-access-56q8n\") pod \"klusterlet-addon-workmgr-5984b86486-sr5c5\" (UID: \"16337e01-6e7f-4222-8e91-ce12fffe034b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"
Jul 29 15:47:45.337607 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.337579    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-gdtj7"]
Jul 29 15:47:45.337739 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.337711    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 15:47:45.405314 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.405220    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d7a2e79-8998-43d4-ba83-a62030f33866-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-668845584b-bk28v\" (UID: \"3d7a2e79-8998-43d4-ba83-a62030f33866\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668845584b-bk28v"
Jul 29 15:47:45.405314 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.405299    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m72d\" (UniqueName: \"kubernetes.io/projected/30ed7de3-8456-43fe-94ea-82e6dbfc4a0b-kube-api-access-5m72d\") pod \"downloads-7c97dc4555-gdtj7\" (UID: \"30ed7de3-8456-43fe-94ea-82e6dbfc4a0b\") " pod="openshift-console/downloads-7c97dc4555-gdtj7"
Jul 29 15:47:45.405529 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.405332    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5l6c\" (UniqueName: \"kubernetes.io/projected/3d7a2e79-8998-43d4-ba83-a62030f33866-kube-api-access-h5l6c\") pod \"managed-serviceaccount-addon-agent-668845584b-bk28v\" (UID: \"3d7a2e79-8998-43d4-ba83-a62030f33866\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668845584b-bk28v"
Jul 29 15:47:45.410329 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.410248    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d7a2e79-8998-43d4-ba83-a62030f33866-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-668845584b-bk28v\" (UID: \"3d7a2e79-8998-43d4-ba83-a62030f33866\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668845584b-bk28v"
Jul 29 15:47:45.412367 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.411897    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78989ff544-62xld" event={"ID":"5dd620e8-a4e4-45e4-be96-0bc62db9a87b","Type":"ContainerStarted","Data":"1e5fdd13436dc63f854ac2b7d88b4bfa5ee78fc740ed2a8736ac72683d740244"}
Jul 29 15:47:45.412367 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.412307    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:47:45.426087 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.425404    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5l6c\" (UniqueName: \"kubernetes.io/projected/3d7a2e79-8998-43d4-ba83-a62030f33866-kube-api-access-h5l6c\") pod \"managed-serviceaccount-addon-agent-668845584b-bk28v\" (UID: \"3d7a2e79-8998-43d4-ba83-a62030f33866\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668845584b-bk28v"
Jul 29 15:47:45.453167 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.452888    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78989ff544-62xld" podStartSLOduration=36.45287064 podStartE2EDuration="36.45287064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:47:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:47:45.452612395 +0000 UTC m=+69.894818656" watchObservedRunningTime="2026-07-29 15:47:45.45287064 +0000 UTC m=+69.895076900"
Jul 29 15:47:45.457879 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.457817    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"
Jul 29 15:47:45.484685 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.484479    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668845584b-bk28v"
Jul 29 15:47:45.506588 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.506539    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m72d\" (UniqueName: \"kubernetes.io/projected/30ed7de3-8456-43fe-94ea-82e6dbfc4a0b-kube-api-access-5m72d\") pod \"downloads-7c97dc4555-gdtj7\" (UID: \"30ed7de3-8456-43fe-94ea-82e6dbfc4a0b\") " pod="openshift-console/downloads-7c97dc4555-gdtj7"
Jul 29 15:47:45.519658 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.519619    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m72d\" (UniqueName: \"kubernetes.io/projected/30ed7de3-8456-43fe-94ea-82e6dbfc4a0b-kube-api-access-5m72d\") pod \"downloads-7c97dc4555-gdtj7\" (UID: \"30ed7de3-8456-43fe-94ea-82e6dbfc4a0b\") " pod="openshift-console/downloads-7c97dc4555-gdtj7"
Jul 29 15:47:45.643757 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.643500    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-gdtj7"
Jul 29 15:47:45.895545 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.895448    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"]
Jul 29 15:47:45.926510 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.926478    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668845584b-bk28v"]
Jul 29 15:47:45.950456 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:45.950420    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-gdtj7"]
Jul 29 15:47:46.693673 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:47:46.693630    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d7a2e79_8998_43d4_ba83_a62030f33866.slice/crio-b49e0dbe308c2440a1b357dcfe43dd67ed21b638299c29f501c91453bf21f528 WatchSource:0}: Error finding container b49e0dbe308c2440a1b357dcfe43dd67ed21b638299c29f501c91453bf21f528: Status 404 returned error can't find the container with id b49e0dbe308c2440a1b357dcfe43dd67ed21b638299c29f501c91453bf21f528
Jul 29 15:47:46.694282 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:47:46.694257    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16337e01_6e7f_4222_8e91_ce12fffe034b.slice/crio-f93f09260c329347e4ff25f302d77ded1b8d9bf7a657e392125768ff572ed883 WatchSource:0}: Error finding container f93f09260c329347e4ff25f302d77ded1b8d9bf7a657e392125768ff572ed883: Status 404 returned error can't find the container with id f93f09260c329347e4ff25f302d77ded1b8d9bf7a657e392125768ff572ed883
Jul 29 15:47:46.695614 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:47:46.695598    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30ed7de3_8456_43fe_94ea_82e6dbfc4a0b.slice/crio-67f1042459d91aef542d5d92936490d96eb73d1dd12f080fcfecc9a35bcfad21 WatchSource:0}: Error finding container 67f1042459d91aef542d5d92936490d96eb73d1dd12f080fcfecc9a35bcfad21: Status 404 returned error can't find the container with id 67f1042459d91aef542d5d92936490d96eb73d1dd12f080fcfecc9a35bcfad21
Jul 29 15:47:47.425268 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.425229    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mszm6" event={"ID":"8aac57b2-7e6a-45ec-8d29-96224fcfc997","Type":"ContainerStarted","Data":"acd4764709704a62a2247202daddfaf3f55c017b6a8d5c81b0914fd6021760f2"}
Jul 29 15:47:47.425436 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.425336    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mszm6" event={"ID":"8aac57b2-7e6a-45ec-8d29-96224fcfc997","Type":"ContainerStarted","Data":"d98542736a13e2abc8366e59e1dfd9e01aa98de1074b8087ab0b255455612e9d"}
Jul 29 15:47:47.426034 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.426005    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mszm6"
Jul 29 15:47:47.427757 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.427731    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-gdtj7" event={"ID":"30ed7de3-8456-43fe-94ea-82e6dbfc4a0b","Type":"ContainerStarted","Data":"67f1042459d91aef542d5d92936490d96eb73d1dd12f080fcfecc9a35bcfad21"}
Jul 29 15:47:47.431641 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.431610    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-grvsv" event={"ID":"902d7a09-dd0e-466d-95e3-a80d69c1dd11","Type":"ContainerStarted","Data":"998112c63ef71dc8b4b918f7a34322c82720e7d1de7b57648fc139c036f7b499"}
Jul 29 15:47:47.431739 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.431667    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-grvsv" event={"ID":"902d7a09-dd0e-466d-95e3-a80d69c1dd11","Type":"ContainerStarted","Data":"4d5e9b456b78da08b80dce0ff30962fd40417be98f10a027dcf35e6d6202fb2c"}
Jul 29 15:47:47.433473 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.433448    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668845584b-bk28v" event={"ID":"3d7a2e79-8998-43d4-ba83-a62030f33866","Type":"ContainerStarted","Data":"b49e0dbe308c2440a1b357dcfe43dd67ed21b638299c29f501c91453bf21f528"}
Jul 29 15:47:47.435084 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.435061    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5" event={"ID":"16337e01-6e7f-4222-8e91-ce12fffe034b","Type":"ContainerStarted","Data":"f93f09260c329347e4ff25f302d77ded1b8d9bf7a657e392125768ff572ed883"}
Jul 29 15:47:47.446620 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.446455    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mszm6" podStartSLOduration=34.833949013 podStartE2EDuration="37.446437102s" totalImagesPullingTime="2.612488089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:47:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:47:47.443228474 +0000 UTC m=+71.885434758" watchObservedRunningTime="2026-07-29 15:47:47.446437102 +0000 UTC m=+71.888643361"
Jul 29 15:47:47.464203 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.464034    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-grvsv" podStartSLOduration=70.105323737 podStartE2EDuration="1m11.464022975s" totalImagesPullingTime="1.358699238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:47:47.46323426 +0000 UTC m=+71.905440519" watchObservedRunningTime="2026-07-29 15:47:47.464022975 +0000 UTC m=+71.906229237"
Jul 29 15:47:47.737094 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.735989    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4xfvm"]
Jul 29 15:47:47.739623 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.739220    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4xfvm"
Jul 29 15:47:47.742428 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.742402    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4n6jh\""
Jul 29 15:47:47.742598 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.742579    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 29 15:47:47.756264 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.756205    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4xfvm"]
Jul 29 15:47:47.825717 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.825597    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/13ca2145-9969-4721-a676-86ce906270c3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-4xfvm\" (UID: \"13ca2145-9969-4721-a676-86ce906270c3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4xfvm"
Jul 29 15:47:47.926622 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:47.926588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/13ca2145-9969-4721-a676-86ce906270c3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-4xfvm\" (UID: \"13ca2145-9969-4721-a676-86ce906270c3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4xfvm"
Jul 29 15:47:47.926802 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:47.926762    2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 29 15:47:47.926862 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:47:47.926828    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13ca2145-9969-4721-a676-86ce906270c3-tls-certificates podName:13ca2145-9969-4721-a676-86ce906270c3 nodeName:}" failed. No retries permitted until 2026-07-29 15:47:48.426806501 +0000 UTC m=+72.869012740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/13ca2145-9969-4721-a676-86ce906270c3-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-4xfvm" (UID: "13ca2145-9969-4721-a676-86ce906270c3") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 29 15:47:48.431261 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:48.431221    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/13ca2145-9969-4721-a676-86ce906270c3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-4xfvm\" (UID: \"13ca2145-9969-4721-a676-86ce906270c3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4xfvm"
Jul 29 15:47:48.434314 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:48.434276    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/13ca2145-9969-4721-a676-86ce906270c3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-4xfvm\" (UID: \"13ca2145-9969-4721-a676-86ce906270c3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4xfvm"
Jul 29 15:47:48.653909 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:48.653859    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4xfvm"
Jul 29 15:47:55.063945 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.063902    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4xfvm"]
Jul 29 15:47:55.071016 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:47:55.070975    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ca2145_9969_4721_a676_86ce906270c3.slice/crio-3a1f22daf5ef64b8e507ce44ee88a05f47b3d018e7327fe8349d33fc60911a61 WatchSource:0}: Error finding container 3a1f22daf5ef64b8e507ce44ee88a05f47b3d018e7327fe8349d33fc60911a61: Status 404 returned error can't find the container with id 3a1f22daf5ef64b8e507ce44ee88a05f47b3d018e7327fe8349d33fc60911a61
Jul 29 15:47:55.470976 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.470941    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2k9gn" event={"ID":"41f96031-45cd-4059-9d82-e277dd98bb71","Type":"ContainerStarted","Data":"d72ba2dc479154234ac17c2ff1bbc58d0d36ef3c900b1aab6fd43be025d849af"}
Jul 29 15:47:55.472696 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.472586    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6w8mg" event={"ID":"fce7281a-8d9e-4cda-a8ba-7480f20da3ef","Type":"ContainerStarted","Data":"2197b30d97a17a375ebf02400292c4da7494ae2f1b75ef72c8adcd6bc1d444f4"}
Jul 29 15:47:55.474760 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.474684    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z6jwx" event={"ID":"c45ef580-96eb-4119-89d3-d4dd468e7917","Type":"ContainerStarted","Data":"28d0299165af8ff5d1994719e0c7f58bfa4da768d1ffc93bb1e59c2163c19fac"}
Jul 29 15:47:55.474760 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.474732    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:47:55.476547 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.476455    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668845584b-bk28v" event={"ID":"3d7a2e79-8998-43d4-ba83-a62030f33866","Type":"ContainerStarted","Data":"78a17840694db7d85416a25f607d52be2e0aa3686262641fad7b7e4be020438e"}
Jul 29 15:47:55.478516 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.478471    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4xfvm" event={"ID":"13ca2145-9969-4721-a676-86ce906270c3","Type":"ContainerStarted","Data":"3a1f22daf5ef64b8e507ce44ee88a05f47b3d018e7327fe8349d33fc60911a61"}
Jul 29 15:47:55.480269 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.480246    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5" event={"ID":"16337e01-6e7f-4222-8e91-ce12fffe034b","Type":"ContainerStarted","Data":"d63ed1ba55859667e0775d5353bbb966b2c5ed124b49b3f43fe6b78774aa88d0"}
Jul 29 15:47:55.480491 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.480454    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"
Jul 29 15:47:55.482496 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.482475    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5"
Jul 29 15:47:55.495162 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.495125    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2k9gn" podStartSLOduration=17.814853844 podStartE2EDuration="37.495111247s" totalImagesPullingTime="19.680257403s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:47:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:47:55.493760448 +0000 UTC m=+79.935966718" watchObservedRunningTime="2026-07-29 15:47:55.495111247 +0000 UTC m=+79.937317505"
Jul 29 15:47:55.536492 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.536440    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5984b86486-sr5c5" podStartSLOduration=2.221375019 podStartE2EDuration="10.536421355s" totalImagesPullingTime="8.315046336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:47:55.535504758 +0000 UTC m=+79.977711019" watchObservedRunningTime="2026-07-29 15:47:55.536421355 +0000 UTC m=+79.978627614"
Jul 29 15:47:55.537644 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.537586    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668845584b-bk28v" podStartSLOduration=2.316348632 podStartE2EDuration="10.537576128s" totalImagesPullingTime="8.221227496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:47:55.515546849 +0000 UTC m=+79.957753109" watchObservedRunningTime="2026-07-29 15:47:55.537576128 +0000 UTC m=+79.979782388"
Jul 29 15:47:55.554533 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.554495    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z6jwx" podStartSLOduration=68.797386766 podStartE2EDuration="1m19.554481361s" totalImagesPullingTime="10.757094595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:47:55.553565091 +0000 UTC m=+79.995771343" watchObservedRunningTime="2026-07-29 15:47:55.554481361 +0000 UTC m=+79.996687644"
Jul 29 15:47:55.574270 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:55.574221    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6w8mg" podStartSLOduration=69.03826992 podStartE2EDuration="1m19.574204031s" totalImagesPullingTime="10.535934111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:47:55.573579461 +0000 UTC m=+80.015785721" watchObservedRunningTime="2026-07-29 15:47:55.574204031 +0000 UTC m=+80.016410295"
Jul 29 15:47:56.486076 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:56.486045    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4xfvm" event={"ID":"13ca2145-9969-4721-a676-86ce906270c3","Type":"ContainerStarted","Data":"47f5bad188c5da29f2041ef0a77a3b227559a75a6e1237cee4648f8529f4a13d"}
Jul 29 15:47:56.512909 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:56.512859    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4xfvm" podStartSLOduration=8.183372053 podStartE2EDuration="9.51284418s" totalImagesPullingTime="1.329472127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:47:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:47:56.512502159 +0000 UTC m=+80.954708416" watchObservedRunningTime="2026-07-29 15:47:56.51284418 +0000 UTC m=+80.955050439"
Jul 29 15:47:57.489544 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:57.489512    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4xfvm"
Jul 29 15:47:57.494955 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:57.494929    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4xfvm"
Jul 29 15:47:58.447383 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:47:58.447353    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mszm6"
Jul 29 15:48:02.174432 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.173478    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"]
Jul 29 15:48:02.228096 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.228053    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s5l4t"]
Jul 29 15:48:02.228473 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.228449    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:02.231723 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.231687    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 29 15:48:02.232316 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.232115    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 29 15:48:02.232316 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.232181    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-jcxjx\""
Jul 29 15:48:02.233268 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.232991    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 15:48:02.233268 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.233191    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 15:48:02.234086 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.233830    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 15:48:02.251746 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.251715    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"]
Jul 29 15:48:02.251878 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.251862    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.256818 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.256396    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 15:48:02.256818 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.256422    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 15:48:02.256818 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.256460    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 15:48:02.256818 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.256694    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hcxj9\""
Jul 29 15:48:02.337482 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.337443    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91fc3e7a-d563-45f7-a927-874c9f3ac6df-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-5mlq9\" (UID: \"91fc3e7a-d563-45f7-a927-874c9f3ac6df\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:02.337482 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.337485    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-sys\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.337725 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.337509    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-accelerators-collector-config\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.337725 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.337540    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l27nd\" (UniqueName: \"kubernetes.io/projected/91fc3e7a-d563-45f7-a927-874c9f3ac6df-kube-api-access-l27nd\") pod \"openshift-state-metrics-b8f49f5c6-5mlq9\" (UID: \"91fc3e7a-d563-45f7-a927-874c9f3ac6df\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:02.337725 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.337567    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-textfile\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.337725 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.337596    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-tls\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.337725 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.337622    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/91fc3e7a-d563-45f7-a927-874c9f3ac6df-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-5mlq9\" (UID: \"91fc3e7a-d563-45f7-a927-874c9f3ac6df\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:02.337725 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.337648    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91fc3e7a-d563-45f7-a927-874c9f3ac6df-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-5mlq9\" (UID: \"91fc3e7a-d563-45f7-a927-874c9f3ac6df\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:02.337725 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.337685    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.337725 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.337716    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-wtmp\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.338166 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.337746    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-root\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.338166 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.337770    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-metrics-client-ca\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.338166 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.337794    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc867\" (UniqueName: \"kubernetes.io/projected/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-kube-api-access-rc867\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.439248 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.439153    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/91fc3e7a-d563-45f7-a927-874c9f3ac6df-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-5mlq9\" (UID: \"91fc3e7a-d563-45f7-a927-874c9f3ac6df\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:02.439248 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.439205    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91fc3e7a-d563-45f7-a927-874c9f3ac6df-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-5mlq9\" (UID: \"91fc3e7a-d563-45f7-a927-874c9f3ac6df\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:02.439248 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.439243    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.439523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.439274    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-wtmp\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.439523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.439301    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-root\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.439523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.439325    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-metrics-client-ca\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.439523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.439347    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc867\" (UniqueName: \"kubernetes.io/projected/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-kube-api-access-rc867\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.439523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.439394    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91fc3e7a-d563-45f7-a927-874c9f3ac6df-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-5mlq9\" (UID: \"91fc3e7a-d563-45f7-a927-874c9f3ac6df\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:02.439523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.439418    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-sys\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.439523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.439440    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-accelerators-collector-config\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.439523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.439474    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l27nd\" (UniqueName: \"kubernetes.io/projected/91fc3e7a-d563-45f7-a927-874c9f3ac6df-kube-api-access-l27nd\") pod \"openshift-state-metrics-b8f49f5c6-5mlq9\" (UID: \"91fc3e7a-d563-45f7-a927-874c9f3ac6df\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:02.439523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.439502    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-textfile\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.440054 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.439533    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-tls\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.440054 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.439773    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-sys\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.440328 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.440306    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-metrics-client-ca\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.440418 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:48:02.440404    2560 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 29 15:48:02.440474 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:48:02.440463    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91fc3e7a-d563-45f7-a927-874c9f3ac6df-openshift-state-metrics-tls podName:91fc3e7a-d563-45f7-a927-874c9f3ac6df nodeName:}" failed. No retries permitted until 2026-07-29 15:48:02.940445098 +0000 UTC m=+87.382651342 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/91fc3e7a-d563-45f7-a927-874c9f3ac6df-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-5mlq9" (UID: "91fc3e7a-d563-45f7-a927-874c9f3ac6df") : secret "openshift-state-metrics-tls" not found
Jul 29 15:48:02.440750 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.440721    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91fc3e7a-d563-45f7-a927-874c9f3ac6df-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-5mlq9\" (UID: \"91fc3e7a-d563-45f7-a927-874c9f3ac6df\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:02.440872 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.440856    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-wtmp\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.442736 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.441258    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-root\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.442736 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.441418    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-accelerators-collector-config\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.442736 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.442350    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-textfile\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.442736 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.442425    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-tls\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.445459 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.445438    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91fc3e7a-d563-45f7-a927-874c9f3ac6df-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-5mlq9\" (UID: \"91fc3e7a-d563-45f7-a927-874c9f3ac6df\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:02.445982 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.445957    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.453238 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.453204    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc867\" (UniqueName: \"kubernetes.io/projected/35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11-kube-api-access-rc867\") pod \"node-exporter-s5l4t\" (UID: \"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11\") " pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.454365 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.454321    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l27nd\" (UniqueName: \"kubernetes.io/projected/91fc3e7a-d563-45f7-a927-874c9f3ac6df-kube-api-access-l27nd\") pod \"openshift-state-metrics-b8f49f5c6-5mlq9\" (UID: \"91fc3e7a-d563-45f7-a927-874c9f3ac6df\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:02.566866 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.566825    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s5l4t"
Jul 29 15:48:02.943697 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.943663    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/91fc3e7a-d563-45f7-a927-874c9f3ac6df-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-5mlq9\" (UID: \"91fc3e7a-d563-45f7-a927-874c9f3ac6df\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:02.946794 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:02.946767    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/91fc3e7a-d563-45f7-a927-874c9f3ac6df-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-5mlq9\" (UID: \"91fc3e7a-d563-45f7-a927-874c9f3ac6df\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:03.141829 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.141792    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"
Jul 29 15:48:03.322457 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.322378    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:48:03.335926 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.335890    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.338902 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.338862    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 15:48:03.339030 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.338946    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 15:48:03.339030 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.338968    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 15:48:03.339233 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.339214    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 15:48:03.339337 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.339260    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-85nxw\""
Jul 29 15:48:03.340539 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.340521    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 15:48:03.340692 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.340669    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 15:48:03.340791 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.340700    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 15:48:03.341897 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.341529    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 15:48:03.341897 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.341725    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 15:48:03.345664 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.345583    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:48:03.447812 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.447769    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-config-volume\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.448053 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.447834    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ddf6857a-337e-46fb-864e-4458054c3387-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.448053 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.447862    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ddf6857a-337e-46fb-864e-4458054c3387-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.448053 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.447901    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ddf6857a-337e-46fb-864e-4458054c3387-config-out\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.448053 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.447941    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrfmt\" (UniqueName: \"kubernetes.io/projected/ddf6857a-337e-46fb-864e-4458054c3387-kube-api-access-wrfmt\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.448053 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.448009    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.448317 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.448067    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.448317 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.448142    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.448317 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.448172    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-web-config\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.448317 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.448195    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddf6857a-337e-46fb-864e-4458054c3387-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.448317 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.448220    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.448317 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.448245    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf6857a-337e-46fb-864e-4458054c3387-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.448317 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.448295    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.549295 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.549259    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.549295 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.549297    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-web-config\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.549590 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.549330    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddf6857a-337e-46fb-864e-4458054c3387-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.549590 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:48:03.549444    2560 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 29 15:48:03.549590 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.549458    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.549590 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.549493    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf6857a-337e-46fb-864e-4458054c3387-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.549590 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:48:03.549525    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-main-tls podName:ddf6857a-337e-46fb-864e-4458054c3387 nodeName:}" failed. No retries permitted until 2026-07-29 15:48:04.049504262 +0000 UTC m=+88.491710515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "ddf6857a-337e-46fb-864e-4458054c3387") : secret "alertmanager-main-tls" not found
Jul 29 15:48:03.549590 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.549564    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.549886 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.549604    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-config-volume\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.549886 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.549638    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ddf6857a-337e-46fb-864e-4458054c3387-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.549886 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.549655    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ddf6857a-337e-46fb-864e-4458054c3387-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.549886 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.549688    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ddf6857a-337e-46fb-864e-4458054c3387-config-out\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.549886 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.549703    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrfmt\" (UniqueName: \"kubernetes.io/projected/ddf6857a-337e-46fb-864e-4458054c3387-kube-api-access-wrfmt\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.549886 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.549723    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.549886 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.549746    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.550233 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.550152    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddf6857a-337e-46fb-864e-4458054c3387-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.551106 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.550334    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf6857a-337e-46fb-864e-4458054c3387-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.551251 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.551173    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ddf6857a-337e-46fb-864e-4458054c3387-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.552655 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.552608    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-web-config\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.552771 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.552740    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ddf6857a-337e-46fb-864e-4458054c3387-config-out\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.553547 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.553509    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.554084 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.554062    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ddf6857a-337e-46fb-864e-4458054c3387-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.554192 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.554116    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.554461 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.554436    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-config-volume\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.554823 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.554784    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.555616 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.555589    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:03.559987 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:03.559943    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrfmt\" (UniqueName: \"kubernetes.io/projected/ddf6857a-337e-46fb-864e-4458054c3387-kube-api-access-wrfmt\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:04.053988 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.053946    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:04.056755 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.056728    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:04.249127 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.249089    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:48:04.290342 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.290313    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6d95c9f985-z99zd"]
Jul 29 15:48:04.315951 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.315911    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.320399 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.320375    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 29 15:48:04.320573 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.320550    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 29 15:48:04.321309 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.321286    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-cchqf\""
Jul 29 15:48:04.321962 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.321502    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 29 15:48:04.321962 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.321511    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 29 15:48:04.321962 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.321692    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d95c9f985-z99zd"]
Jul 29 15:48:04.321962 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.321728    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1rp0n5p5fhkme\""
Jul 29 15:48:04.322301 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.322283    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 29 15:48:04.458015 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.457979    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d88bed02-586a-40a1-babf-b48a43f918a7-metrics-client-ca\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.458506 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.458042    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.458506 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.458150    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pqbw\" (UniqueName: \"kubernetes.io/projected/d88bed02-586a-40a1-babf-b48a43f918a7-kube-api-access-5pqbw\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.458506 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.458184    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.458506 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.458218    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-tls\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.458506 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.458295    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.458506 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.458337    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-grpc-tls\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.458506 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.458375    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.559533 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.559490    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.559533 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.559546    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-tls\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.559826 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.559586    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.559826 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.559612    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-grpc-tls\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.559826 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.559644    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.560026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.559848    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d88bed02-586a-40a1-babf-b48a43f918a7-metrics-client-ca\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.560026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.559903    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.560026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.559980    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pqbw\" (UniqueName: \"kubernetes.io/projected/d88bed02-586a-40a1-babf-b48a43f918a7-kube-api-access-5pqbw\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.560707 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.560658    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d88bed02-586a-40a1-babf-b48a43f918a7-metrics-client-ca\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.562909 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.562659    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.562909 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.562854    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-tls\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.563226 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.563186    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-grpc-tls\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.563384 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.563356    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.563517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.563494    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.563591 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.563548    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d88bed02-586a-40a1-babf-b48a43f918a7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.570017 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.569947    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pqbw\" (UniqueName: \"kubernetes.io/projected/d88bed02-586a-40a1-babf-b48a43f918a7-kube-api-access-5pqbw\") pod \"thanos-querier-6d95c9f985-z99zd\" (UID: \"d88bed02-586a-40a1-babf-b48a43f918a7\") " pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:04.628429 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:04.628399    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:06.423822 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.423792    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:48:06.621767 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.621723    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-b85954cdf-gnmfg"]
Jul 29 15:48:06.650171 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.650143    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b85954cdf-gnmfg"]
Jul 29 15:48:06.650341 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.650252    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.653713 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.653687    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bgmmk\""
Jul 29 15:48:06.653713 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.653708    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 29 15:48:06.653946 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.653773    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2kj9snt2n8kle\""
Jul 29 15:48:06.653946 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.653693    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 29 15:48:06.654082 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.654041    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 15:48:06.654297 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.654282    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 29 15:48:06.781560 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.781487    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e0356d1c-c664-4886-afab-0dda476cef3b-audit-log\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.781560 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.781519    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0356d1c-c664-4886-afab-0dda476cef3b-client-ca-bundle\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.781560 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.781538    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e0356d1c-c664-4886-afab-0dda476cef3b-metrics-server-audit-profiles\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.781560 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.781561    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e0356d1c-c664-4886-afab-0dda476cef3b-secret-metrics-server-client-certs\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.781875 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.781651    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e0356d1c-c664-4886-afab-0dda476cef3b-secret-metrics-server-tls\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.781875 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.781706    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0356d1c-c664-4886-afab-0dda476cef3b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.781875 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.781771    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6759w\" (UniqueName: \"kubernetes.io/projected/e0356d1c-c664-4886-afab-0dda476cef3b-kube-api-access-6759w\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.842970 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:48:06.842939    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35eb4f03_4ed5_4bf6_9b47_ce47e30f9d11.slice/crio-02fac91ab455a59976fdb20cf3c4cb3f28bf1a77d873206dec70d307d264902d WatchSource:0}: Error finding container 02fac91ab455a59976fdb20cf3c4cb3f28bf1a77d873206dec70d307d264902d: Status 404 returned error can't find the container with id 02fac91ab455a59976fdb20cf3c4cb3f28bf1a77d873206dec70d307d264902d
Jul 29 15:48:06.885486 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.883706    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6759w\" (UniqueName: \"kubernetes.io/projected/e0356d1c-c664-4886-afab-0dda476cef3b-kube-api-access-6759w\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.885486 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.883803    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e0356d1c-c664-4886-afab-0dda476cef3b-audit-log\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.885486 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.883829    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0356d1c-c664-4886-afab-0dda476cef3b-client-ca-bundle\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.885486 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.883860    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e0356d1c-c664-4886-afab-0dda476cef3b-metrics-server-audit-profiles\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.885486 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.883896    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e0356d1c-c664-4886-afab-0dda476cef3b-secret-metrics-server-client-certs\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.885486 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.883952    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e0356d1c-c664-4886-afab-0dda476cef3b-secret-metrics-server-tls\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.885486 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.883984    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0356d1c-c664-4886-afab-0dda476cef3b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.885486 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.884731    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0356d1c-c664-4886-afab-0dda476cef3b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.885486 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.885169    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e0356d1c-c664-4886-afab-0dda476cef3b-audit-log\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.886408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.885735    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e0356d1c-c664-4886-afab-0dda476cef3b-metrics-server-audit-profiles\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.888484 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.888435    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0356d1c-c664-4886-afab-0dda476cef3b-client-ca-bundle\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.888735 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.888711    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e0356d1c-c664-4886-afab-0dda476cef3b-secret-metrics-server-tls\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.888975 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.888903    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e0356d1c-c664-4886-afab-0dda476cef3b-secret-metrics-server-client-certs\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.895537 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.895483    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6759w\" (UniqueName: \"kubernetes.io/projected/e0356d1c-c664-4886-afab-0dda476cef3b-kube-api-access-6759w\") pod \"metrics-server-b85954cdf-gnmfg\" (UID: \"e0356d1c-c664-4886-afab-0dda476cef3b\") " pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.961480 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.961184    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:06.998560 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:06.997896    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d95c9f985-z99zd"]
Jul 29 15:48:07.020874 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.020777    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:48:07.023543 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:48:07.023514    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf6857a_337e_46fb_864e_4458054c3387.slice/crio-df516bb9a5bace838632f36676f993ba131f28d21975e0001af7162a8e532418 WatchSource:0}: Error finding container df516bb9a5bace838632f36676f993ba131f28d21975e0001af7162a8e532418: Status 404 returned error can't find the container with id df516bb9a5bace838632f36676f993ba131f28d21975e0001af7162a8e532418
Jul 29 15:48:07.036389 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.036365    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9"]
Jul 29 15:48:07.113257 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.113231    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b85954cdf-gnmfg"]
Jul 29 15:48:07.216256 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:48:07.216224    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91fc3e7a_d563_45f7_a927_874c9f3ac6df.slice/crio-04c6233094fe0412ffc1c2ef4dadc5096044470019b7c9e12005799994f96bfb WatchSource:0}: Error finding container 04c6233094fe0412ffc1c2ef4dadc5096044470019b7c9e12005799994f96bfb: Status 404 returned error can't find the container with id 04c6233094fe0412ffc1c2ef4dadc5096044470019b7c9e12005799994f96bfb
Jul 29 15:48:07.216936 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:48:07.216885    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0356d1c_c664_4886_afab_0dda476cef3b.slice/crio-41221ca7049fe864d1564459edfad184d83c488d119509a39abf999f2f9968a9 WatchSource:0}: Error finding container 41221ca7049fe864d1564459edfad184d83c488d119509a39abf999f2f9968a9: Status 404 returned error can't find the container with id 41221ca7049fe864d1564459edfad184d83c488d119509a39abf999f2f9968a9
Jul 29 15:48:07.531210 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.530992    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-gdtj7" event={"ID":"30ed7de3-8456-43fe-94ea-82e6dbfc4a0b","Type":"ContainerStarted","Data":"238a6cdae7dc557f1c58cd37171db2ed04efb9c81620b9c5cbe8c661694c233b"}
Jul 29 15:48:07.531669 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.531530    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-gdtj7"
Jul 29 15:48:07.533628 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.533130    2560 patch_prober.go:28] interesting pod/downloads-7c97dc4555-gdtj7 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.13:8080/\": dial tcp 10.133.0.13:8080: connect: connection refused" start-of-body=
Jul 29 15:48:07.533628 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.533188    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-gdtj7" podUID="30ed7de3-8456-43fe-94ea-82e6dbfc4a0b" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.13:8080/\": dial tcp 10.133.0.13:8080: connect: connection refused"
Jul 29 15:48:07.534147 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.534115    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg" event={"ID":"e0356d1c-c664-4886-afab-0dda476cef3b","Type":"ContainerStarted","Data":"41221ca7049fe864d1564459edfad184d83c488d119509a39abf999f2f9968a9"}
Jul 29 15:48:07.537038 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.537013    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9" event={"ID":"91fc3e7a-d563-45f7-a927-874c9f3ac6df","Type":"ContainerStarted","Data":"cec735f65ccd18b47ca912644d58ad66ccf763d8112110080ac9bfc90c6b1c0d"}
Jul 29 15:48:07.537169 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.537074    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9" event={"ID":"91fc3e7a-d563-45f7-a927-874c9f3ac6df","Type":"ContainerStarted","Data":"0d77778ff29c746db3ff90de772e446243667fe424387543804780b1b19ed55f"}
Jul 29 15:48:07.537169 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.537104    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9" event={"ID":"91fc3e7a-d563-45f7-a927-874c9f3ac6df","Type":"ContainerStarted","Data":"04c6233094fe0412ffc1c2ef4dadc5096044470019b7c9e12005799994f96bfb"}
Jul 29 15:48:07.538603 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.538475    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd" event={"ID":"d88bed02-586a-40a1-babf-b48a43f918a7","Type":"ContainerStarted","Data":"ee356ab23e6b205a0bad66023cb32ca1cf813a726d9aa063422dc4ec7b917d59"}
Jul 29 15:48:07.539971 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.539947    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerStarted","Data":"df516bb9a5bace838632f36676f993ba131f28d21975e0001af7162a8e532418"}
Jul 29 15:48:07.541520 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.541463    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s5l4t" event={"ID":"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11","Type":"ContainerStarted","Data":"02fac91ab455a59976fdb20cf3c4cb3f28bf1a77d873206dec70d307d264902d"}
Jul 29 15:48:07.554515 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:07.554445    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-gdtj7" podStartSLOduration=1.95312712 podStartE2EDuration="22.554426365s" totalImagesPullingTime="20.601299245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:48:07.554373975 +0000 UTC m=+91.996580239" watchObservedRunningTime="2026-07-29 15:48:07.554426365 +0000 UTC m=+91.996632625"
Jul 29 15:48:08.559670 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:08.559427    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-gdtj7"
Jul 29 15:48:11.449749 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.449625    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-78989ff544-62xld" podUID="5dd620e8-a4e4-45e4-be96-0bc62db9a87b" containerName="registry" containerID="cri-o://1e5fdd13436dc63f854ac2b7d88b4bfa5ee78fc740ed2a8736ac72683d740244" gracePeriod=30
Jul 29 15:48:11.566694 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.566666    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9" event={"ID":"91fc3e7a-d563-45f7-a927-874c9f3ac6df","Type":"ContainerStarted","Data":"914d3e2a94d243b01b5b24e0ee3bc20ef7a168458dbc3b5535143697e468ae93"}
Jul 29 15:48:11.568882 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.568828    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd" event={"ID":"d88bed02-586a-40a1-babf-b48a43f918a7","Type":"ContainerStarted","Data":"69e31205658135373fb5053a6d652e96775a47eb2b6c70e20f13f662d14c21be"}
Jul 29 15:48:11.571932 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.571887    2560 generic.go:356] "Generic (PLEG): container finished" podID="ddf6857a-337e-46fb-864e-4458054c3387" containerID="c26d3981be2a2812d2f9df7e6135a54b81403c02f8254761cdaf78e818b8f516" exitCode=0
Jul 29 15:48:11.572031 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.572014    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerDied","Data":"c26d3981be2a2812d2f9df7e6135a54b81403c02f8254761cdaf78e818b8f516"}
Jul 29 15:48:11.578818 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.578192    2560 generic.go:356] "Generic (PLEG): container finished" podID="35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11" containerID="90cb14ce6bcd995eb87992d922849f878889fc94ecc1605c57bf083100d901a3" exitCode=0
Jul 29 15:48:11.578818 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.578484    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s5l4t" event={"ID":"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11","Type":"ContainerDied","Data":"90cb14ce6bcd995eb87992d922849f878889fc94ecc1605c57bf083100d901a3"}
Jul 29 15:48:11.585968 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.585293    2560 generic.go:356] "Generic (PLEG): container finished" podID="5dd620e8-a4e4-45e4-be96-0bc62db9a87b" containerID="1e5fdd13436dc63f854ac2b7d88b4bfa5ee78fc740ed2a8736ac72683d740244" exitCode=0
Jul 29 15:48:11.585968 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.585357    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78989ff544-62xld" event={"ID":"5dd620e8-a4e4-45e4-be96-0bc62db9a87b","Type":"ContainerDied","Data":"1e5fdd13436dc63f854ac2b7d88b4bfa5ee78fc740ed2a8736ac72683d740244"}
Jul 29 15:48:11.586204 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.586158    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5mlq9" podStartSLOduration=5.964853558 podStartE2EDuration="9.586139248s" totalImagesPullingTime="3.62128569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:48:11.585634658 +0000 UTC m=+96.027840919" watchObservedRunningTime="2026-07-29 15:48:11.586139248 +0000 UTC m=+96.028345507"
Jul 29 15:48:11.757477 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.757272    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:48:11.830469 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.830442    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-image-registry-private-configuration\") pod \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") "
Jul 29 15:48:11.830591 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.830481    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-bound-sa-token\") pod \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") "
Jul 29 15:48:11.830591 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.830522    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-ca-trust-extracted\") pod \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") "
Jul 29 15:48:11.830591 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.830568    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls\") pod \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") "
Jul 29 15:48:11.830756 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.830611    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-trusted-ca\") pod \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") "
Jul 29 15:48:11.830756 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.830645    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6kc7\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-kube-api-access-m6kc7\") pod \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") "
Jul 29 15:48:11.830756 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.830672    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-certificates\") pod \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") "
Jul 29 15:48:11.830756 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.830708    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-installation-pull-secrets\") pod \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\" (UID: \"5dd620e8-a4e4-45e4-be96-0bc62db9a87b\") "
Jul 29 15:48:11.832103 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.832072    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5dd620e8-a4e4-45e4-be96-0bc62db9a87b" (UID: "5dd620e8-a4e4-45e4-be96-0bc62db9a87b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:48:11.832406 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.832385    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5dd620e8-a4e4-45e4-be96-0bc62db9a87b" (UID: "5dd620e8-a4e4-45e4-be96-0bc62db9a87b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:48:11.834094 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.834050    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5dd620e8-a4e4-45e4-be96-0bc62db9a87b" (UID: "5dd620e8-a4e4-45e4-be96-0bc62db9a87b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:48:11.834225 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.834205    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-kube-api-access-m6kc7" (OuterVolumeSpecName: "kube-api-access-m6kc7") pod "5dd620e8-a4e4-45e4-be96-0bc62db9a87b" (UID: "5dd620e8-a4e4-45e4-be96-0bc62db9a87b"). InnerVolumeSpecName "kube-api-access-m6kc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:48:11.834325 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.834293    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5dd620e8-a4e4-45e4-be96-0bc62db9a87b" (UID: "5dd620e8-a4e4-45e4-be96-0bc62db9a87b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:48:11.835966 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.835898    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5dd620e8-a4e4-45e4-be96-0bc62db9a87b" (UID: "5dd620e8-a4e4-45e4-be96-0bc62db9a87b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:48:11.836236 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.836200    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5dd620e8-a4e4-45e4-be96-0bc62db9a87b" (UID: "5dd620e8-a4e4-45e4-be96-0bc62db9a87b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:48:11.842537 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.842503    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5dd620e8-a4e4-45e4-be96-0bc62db9a87b" (UID: "5dd620e8-a4e4-45e4-be96-0bc62db9a87b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:48:11.931586 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.931552    2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-bound-sa-token\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:48:11.931586 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.931590    2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-ca-trust-extracted\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:48:11.931798 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.931604    2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-tls\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:48:11.931798 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.931613    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-trusted-ca\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:48:11.931798 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.931621    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6kc7\" (UniqueName: \"kubernetes.io/projected/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-kube-api-access-m6kc7\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:48:11.931798 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.931632    2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-registry-certificates\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:48:11.931798 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.931646    2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-installation-pull-secrets\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:48:11.931798 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:11.931661    2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dd620e8-a4e4-45e4-be96-0bc62db9a87b-image-registry-private-configuration\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:48:12.593330 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:12.593289    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s5l4t" event={"ID":"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11","Type":"ContainerStarted","Data":"d7c7bc486afef6bd971ebd94e8e67ab33fc768639fc77fddcacaccf6bd6a9cad"}
Jul 29 15:48:12.593330 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:12.593354    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s5l4t" event={"ID":"35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11","Type":"ContainerStarted","Data":"6c0bd054271dbfbd239262f074b6d1e352ce4daa9b1f3c847dc73893edbb76c9"}
Jul 29 15:48:12.595461 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:12.595416    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78989ff544-62xld" event={"ID":"5dd620e8-a4e4-45e4-be96-0bc62db9a87b","Type":"ContainerDied","Data":"40613f87cb4a5cc973a46cbb4f5d5253f1391b67d547adc0d4395e9c1a91bb2b"}
Jul 29 15:48:12.595610 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:12.595470    2560 scope.go:117] "RemoveContainer" containerID="1e5fdd13436dc63f854ac2b7d88b4bfa5ee78fc740ed2a8736ac72683d740244"
Jul 29 15:48:12.595657 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:12.595631    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78989ff544-62xld"
Jul 29 15:48:12.602164 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:12.601634    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg" event={"ID":"e0356d1c-c664-4886-afab-0dda476cef3b","Type":"ContainerStarted","Data":"2c608f88059f46363de090a005ed7676502091e7bec3ac837fa8c851a6122edc"}
Jul 29 15:48:12.607991 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:12.607840    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd" event={"ID":"d88bed02-586a-40a1-babf-b48a43f918a7","Type":"ContainerStarted","Data":"3e6e3d0de9d5cc68e4268d4995b78961410cc1e7614e16d585860b848c0718ed"}
Jul 29 15:48:12.607991 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:12.607934    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd" event={"ID":"d88bed02-586a-40a1-babf-b48a43f918a7","Type":"ContainerStarted","Data":"3f956d3d1704afcffa54cfde3514af40649af4456998caf87f25959b92e9b3f3"}
Jul 29 15:48:12.618592 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:12.618553    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s5l4t" podStartSLOduration=6.292632996 podStartE2EDuration="10.618536915s" totalImagesPullingTime="4.226868906s" totalInitContainerRuntime="99.035013ms" isStatefulPod=true podCreationTimestamp="2026-07-29 15:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:48:12.616005995 +0000 UTC m=+97.058212255" watchObservedRunningTime="2026-07-29 15:48:12.618536915 +0000 UTC m=+97.060743173"
Jul 29 15:48:12.639754 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:12.639369    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg" podStartSLOduration=2.4648630750000002 podStartE2EDuration="6.639349446s" totalImagesPullingTime="4.174486371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:48:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:48:12.638673983 +0000 UTC m=+97.080880243" watchObservedRunningTime="2026-07-29 15:48:12.639349446 +0000 UTC m=+97.081555706"
Jul 29 15:48:12.662361 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:12.662288    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78989ff544-62xld"]
Jul 29 15:48:12.671796 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:12.671738    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78989ff544-62xld"]
Jul 29 15:48:14.043317 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:14.043288    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dd620e8-a4e4-45e4-be96-0bc62db9a87b" path="/var/lib/kubelet/pods/5dd620e8-a4e4-45e4-be96-0bc62db9a87b/volumes"
Jul 29 15:48:14.620563 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:14.620359    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd" event={"ID":"d88bed02-586a-40a1-babf-b48a43f918a7","Type":"ContainerStarted","Data":"51276eb023311949b1eb8ecf4a5a46e8d77425ccf172c7d5d5eb3c2155223eb5"}
Jul 29 15:48:14.620563 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:14.620436    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd" event={"ID":"d88bed02-586a-40a1-babf-b48a43f918a7","Type":"ContainerStarted","Data":"db91034f155b81324b2d106bf0c3399b6337a9d156fc7e7cbb4f9f37aea962a1"}
Jul 29 15:48:14.620563 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:14.620476    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd" event={"ID":"d88bed02-586a-40a1-babf-b48a43f918a7","Type":"ContainerStarted","Data":"75b28cf1cc0f71bee7e361ad2f44d30365e42fc720e13dd4e6b60b4eb9d3bc30"}
Jul 29 15:48:14.620563 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:14.620519    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:14.624089 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:14.624056    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerStarted","Data":"3911c5bb645dacfba8f644e21908f8297616cdce549dc545c957db201b1fd2a4"}
Jul 29 15:48:14.624183 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:14.624129    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerStarted","Data":"37dfc12134c52829e36640edd25eb59972c807b8a19a13bb4bc72e7e436e5108"}
Jul 29 15:48:14.624183 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:14.624171    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerStarted","Data":"e6893c098c7ecc010fe05c1ea8312845e28f9a2100a0ee4624dd3559ecdf26c3"}
Jul 29 15:48:14.624262 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:14.624213    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerStarted","Data":"273fe5de50af971c5440eea8778638ae3691a881f1984ded917f1498787a8e8e"}
Jul 29 15:48:14.651240 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:14.651192    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd" podStartSLOduration=4.614387847 podStartE2EDuration="10.651174126s" totalImagesPullingTime="6.036786279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:48:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:48:14.64934825 +0000 UTC m=+99.091554511" watchObservedRunningTime="2026-07-29 15:48:14.651174126 +0000 UTC m=+99.093380386"
Jul 29 15:48:15.633693 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:15.633654    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerStarted","Data":"aa143f859c4c29d7c6a033edf15a5091eefb6eb2da60c18523f2acbdfca4fd59"}
Jul 29 15:48:15.634192 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:15.633745    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerStarted","Data":"c156c8304d0c3dbd1aa9ba6219858ee8ef45f9e5577b6664d501247eb48ce9a3"}
Jul 29 15:48:15.663641 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:15.663555    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.133370672 podStartE2EDuration="12.663533839s" totalImagesPullingTime="6.508673987s" totalInitContainerRuntime="21.48918ms" isStatefulPod=false podCreationTimestamp="2026-07-29 15:48:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:48:15.661379896 +0000 UTC m=+100.103586176" watchObservedRunningTime="2026-07-29 15:48:15.663533839 +0000 UTC m=+100.105740099"
Jul 29 15:48:20.640148 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:20.640120    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6d95c9f985-z99zd"
Jul 29 15:48:24.437688 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:24.437658    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6wtqz_4f7e28c6-0d23-445a-9b23-725d4dc65be1/serve-healthcheck-canary/0.log"
Jul 29 15:48:26.489678 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:26.489648    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z6jwx"
Jul 29 15:48:26.961699 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:26.961669    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:26.961850 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:26.961735    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:46.966961 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:46.966906    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:48:46.970771 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:48:46.970746    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-b85954cdf-gnmfg"
Jul 29 15:49:22.609497 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.609466    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:49:22.610030 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.609755    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="alertmanager" containerID="cri-o://273fe5de50af971c5440eea8778638ae3691a881f1984ded917f1498787a8e8e" gracePeriod=120
Jul 29 15:49:22.610030 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.609822    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="kube-rbac-proxy-metric" containerID="cri-o://c156c8304d0c3dbd1aa9ba6219858ee8ef45f9e5577b6664d501247eb48ce9a3" gracePeriod=120
Jul 29 15:49:22.610030 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.609858    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="kube-rbac-proxy-web" containerID="cri-o://37dfc12134c52829e36640edd25eb59972c807b8a19a13bb4bc72e7e436e5108" gracePeriod=120
Jul 29 15:49:22.610030 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.609894    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="prom-label-proxy" containerID="cri-o://aa143f859c4c29d7c6a033edf15a5091eefb6eb2da60c18523f2acbdfca4fd59" gracePeriod=120
Jul 29 15:49:22.610030 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.609949    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="kube-rbac-proxy" containerID="cri-o://3911c5bb645dacfba8f644e21908f8297616cdce549dc545c957db201b1fd2a4" gracePeriod=120
Jul 29 15:49:22.610030 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.609890    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="config-reloader" containerID="cri-o://e6893c098c7ecc010fe05c1ea8312845e28f9a2100a0ee4624dd3559ecdf26c3" gracePeriod=120
Jul 29 15:49:22.888430 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.888347    2560 generic.go:356] "Generic (PLEG): container finished" podID="ddf6857a-337e-46fb-864e-4458054c3387" containerID="aa143f859c4c29d7c6a033edf15a5091eefb6eb2da60c18523f2acbdfca4fd59" exitCode=0
Jul 29 15:49:22.888430 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.888370    2560 generic.go:356] "Generic (PLEG): container finished" podID="ddf6857a-337e-46fb-864e-4458054c3387" containerID="c156c8304d0c3dbd1aa9ba6219858ee8ef45f9e5577b6664d501247eb48ce9a3" exitCode=0
Jul 29 15:49:22.888430 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.888378    2560 generic.go:356] "Generic (PLEG): container finished" podID="ddf6857a-337e-46fb-864e-4458054c3387" containerID="3911c5bb645dacfba8f644e21908f8297616cdce549dc545c957db201b1fd2a4" exitCode=0
Jul 29 15:49:22.888430 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.888387    2560 generic.go:356] "Generic (PLEG): container finished" podID="ddf6857a-337e-46fb-864e-4458054c3387" containerID="e6893c098c7ecc010fe05c1ea8312845e28f9a2100a0ee4624dd3559ecdf26c3" exitCode=0
Jul 29 15:49:22.888430 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.888394    2560 generic.go:356] "Generic (PLEG): container finished" podID="ddf6857a-337e-46fb-864e-4458054c3387" containerID="273fe5de50af971c5440eea8778638ae3691a881f1984ded917f1498787a8e8e" exitCode=0
Jul 29 15:49:22.888430 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.888402    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerDied","Data":"aa143f859c4c29d7c6a033edf15a5091eefb6eb2da60c18523f2acbdfca4fd59"}
Jul 29 15:49:22.888722 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.888461    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerDied","Data":"c156c8304d0c3dbd1aa9ba6219858ee8ef45f9e5577b6664d501247eb48ce9a3"}
Jul 29 15:49:22.888722 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.888486    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerDied","Data":"3911c5bb645dacfba8f644e21908f8297616cdce549dc545c957db201b1fd2a4"}
Jul 29 15:49:22.888722 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.888511    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerDied","Data":"e6893c098c7ecc010fe05c1ea8312845e28f9a2100a0ee4624dd3559ecdf26c3"}
Jul 29 15:49:22.888722 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:22.888537    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerDied","Data":"273fe5de50af971c5440eea8778638ae3691a881f1984ded917f1498787a8e8e"}
Jul 29 15:49:23.850263 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.850233    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:23.897063 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.897033    2560 generic.go:356] "Generic (PLEG): container finished" podID="ddf6857a-337e-46fb-864e-4458054c3387" containerID="37dfc12134c52829e36640edd25eb59972c807b8a19a13bb4bc72e7e436e5108" exitCode=0
Jul 29 15:49:23.897227 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.897110    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerDied","Data":"37dfc12134c52829e36640edd25eb59972c807b8a19a13bb4bc72e7e436e5108"}
Jul 29 15:49:23.897227 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.897133    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:23.897227 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.897205    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf6857a-337e-46fb-864e-4458054c3387","Type":"ContainerDied","Data":"df516bb9a5bace838632f36676f993ba131f28d21975e0001af7162a8e532418"}
Jul 29 15:49:23.897446 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.897240    2560 scope.go:117] "RemoveContainer" containerID="aa143f859c4c29d7c6a033edf15a5091eefb6eb2da60c18523f2acbdfca4fd59"
Jul 29 15:49:23.899466 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.899445    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-web-config\") pod \"ddf6857a-337e-46fb-864e-4458054c3387\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") "
Jul 29 15:49:23.899561 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.899489    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-cluster-tls-config\") pod \"ddf6857a-337e-46fb-864e-4458054c3387\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") "
Jul 29 15:49:23.899561 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.899527    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-main-tls\") pod \"ddf6857a-337e-46fb-864e-4458054c3387\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") "
Jul 29 15:49:23.899669 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.899558    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy-web\") pod \"ddf6857a-337e-46fb-864e-4458054c3387\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") "
Jul 29 15:49:23.899669 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.899589    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy\") pod \"ddf6857a-337e-46fb-864e-4458054c3387\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") "
Jul 29 15:49:23.899669 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.899620    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ddf6857a-337e-46fb-864e-4458054c3387\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") "
Jul 29 15:49:23.899669 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.899650    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf6857a-337e-46fb-864e-4458054c3387-alertmanager-trusted-ca-bundle\") pod \"ddf6857a-337e-46fb-864e-4458054c3387\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") "
Jul 29 15:49:23.899880 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.899681    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrfmt\" (UniqueName: \"kubernetes.io/projected/ddf6857a-337e-46fb-864e-4458054c3387-kube-api-access-wrfmt\") pod \"ddf6857a-337e-46fb-864e-4458054c3387\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") "
Jul 29 15:49:23.899880 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.899728    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ddf6857a-337e-46fb-864e-4458054c3387-alertmanager-main-db\") pod \"ddf6857a-337e-46fb-864e-4458054c3387\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") "
Jul 29 15:49:23.899880 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.899764    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddf6857a-337e-46fb-864e-4458054c3387-metrics-client-ca\") pod \"ddf6857a-337e-46fb-864e-4458054c3387\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") "
Jul 29 15:49:23.899880 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.899803    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ddf6857a-337e-46fb-864e-4458054c3387-tls-assets\") pod \"ddf6857a-337e-46fb-864e-4458054c3387\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") "
Jul 29 15:49:23.899880 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.899828    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-config-volume\") pod \"ddf6857a-337e-46fb-864e-4458054c3387\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") "
Jul 29 15:49:23.899880 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.899853    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ddf6857a-337e-46fb-864e-4458054c3387-config-out\") pod \"ddf6857a-337e-46fb-864e-4458054c3387\" (UID: \"ddf6857a-337e-46fb-864e-4458054c3387\") "
Jul 29 15:49:23.900443 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.900408    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf6857a-337e-46fb-864e-4458054c3387-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "ddf6857a-337e-46fb-864e-4458054c3387" (UID: "ddf6857a-337e-46fb-864e-4458054c3387"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:49:23.902055 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.900897    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddf6857a-337e-46fb-864e-4458054c3387-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "ddf6857a-337e-46fb-864e-4458054c3387" (UID: "ddf6857a-337e-46fb-864e-4458054c3387"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:49:23.902757 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.902451    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddf6857a-337e-46fb-864e-4458054c3387-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "ddf6857a-337e-46fb-864e-4458054c3387" (UID: "ddf6857a-337e-46fb-864e-4458054c3387"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:49:23.904440 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.904403    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf6857a-337e-46fb-864e-4458054c3387-config-out" (OuterVolumeSpecName: "config-out") pod "ddf6857a-337e-46fb-864e-4458054c3387" (UID: "ddf6857a-337e-46fb-864e-4458054c3387"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:49:23.904961 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.904906    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "ddf6857a-337e-46fb-864e-4458054c3387" (UID: "ddf6857a-337e-46fb-864e-4458054c3387"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:49:23.905246 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.905201    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "ddf6857a-337e-46fb-864e-4458054c3387" (UID: "ddf6857a-337e-46fb-864e-4458054c3387"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:49:23.905393 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.905365    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "ddf6857a-337e-46fb-864e-4458054c3387" (UID: "ddf6857a-337e-46fb-864e-4458054c3387"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:49:23.905814 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.905773    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "ddf6857a-337e-46fb-864e-4458054c3387" (UID: "ddf6857a-337e-46fb-864e-4458054c3387"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:49:23.906080 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.906044    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddf6857a-337e-46fb-864e-4458054c3387-kube-api-access-wrfmt" (OuterVolumeSpecName: "kube-api-access-wrfmt") pod "ddf6857a-337e-46fb-864e-4458054c3387" (UID: "ddf6857a-337e-46fb-864e-4458054c3387"). InnerVolumeSpecName "kube-api-access-wrfmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:49:23.906345 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.906320    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-config-volume" (OuterVolumeSpecName: "config-volume") pod "ddf6857a-337e-46fb-864e-4458054c3387" (UID: "ddf6857a-337e-46fb-864e-4458054c3387"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:49:23.907351 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.907328    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddf6857a-337e-46fb-864e-4458054c3387-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ddf6857a-337e-46fb-864e-4458054c3387" (UID: "ddf6857a-337e-46fb-864e-4458054c3387"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:49:23.908894 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.908876    2560 scope.go:117] "RemoveContainer" containerID="c156c8304d0c3dbd1aa9ba6219858ee8ef45f9e5577b6664d501247eb48ce9a3"
Jul 29 15:49:23.909910 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.909818    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "ddf6857a-337e-46fb-864e-4458054c3387" (UID: "ddf6857a-337e-46fb-864e-4458054c3387"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:49:23.915247 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.915228    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-web-config" (OuterVolumeSpecName: "web-config") pod "ddf6857a-337e-46fb-864e-4458054c3387" (UID: "ddf6857a-337e-46fb-864e-4458054c3387"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:49:23.922699 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.922679    2560 scope.go:117] "RemoveContainer" containerID="3911c5bb645dacfba8f644e21908f8297616cdce549dc545c957db201b1fd2a4"
Jul 29 15:49:23.929606 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.929592    2560 scope.go:117] "RemoveContainer" containerID="37dfc12134c52829e36640edd25eb59972c807b8a19a13bb4bc72e7e436e5108"
Jul 29 15:49:23.935859 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.935842    2560 scope.go:117] "RemoveContainer" containerID="e6893c098c7ecc010fe05c1ea8312845e28f9a2100a0ee4624dd3559ecdf26c3"
Jul 29 15:49:23.942525 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.942510    2560 scope.go:117] "RemoveContainer" containerID="273fe5de50af971c5440eea8778638ae3691a881f1984ded917f1498787a8e8e"
Jul 29 15:49:23.948960 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.948942    2560 scope.go:117] "RemoveContainer" containerID="c26d3981be2a2812d2f9df7e6135a54b81403c02f8254761cdaf78e818b8f516"
Jul 29 15:49:23.955409 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.955395    2560 scope.go:117] "RemoveContainer" containerID="aa143f859c4c29d7c6a033edf15a5091eefb6eb2da60c18523f2acbdfca4fd59"
Jul 29 15:49:23.955670 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:49:23.955646    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa143f859c4c29d7c6a033edf15a5091eefb6eb2da60c18523f2acbdfca4fd59\": container with ID starting with aa143f859c4c29d7c6a033edf15a5091eefb6eb2da60c18523f2acbdfca4fd59 not found: ID does not exist" containerID="aa143f859c4c29d7c6a033edf15a5091eefb6eb2da60c18523f2acbdfca4fd59"
Jul 29 15:49:23.955753 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.955676    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa143f859c4c29d7c6a033edf15a5091eefb6eb2da60c18523f2acbdfca4fd59"} err="failed to get container status \"aa143f859c4c29d7c6a033edf15a5091eefb6eb2da60c18523f2acbdfca4fd59\": rpc error: code = NotFound desc = could not find container \"aa143f859c4c29d7c6a033edf15a5091eefb6eb2da60c18523f2acbdfca4fd59\": container with ID starting with aa143f859c4c29d7c6a033edf15a5091eefb6eb2da60c18523f2acbdfca4fd59 not found: ID does not exist"
Jul 29 15:49:23.955753 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.955718    2560 scope.go:117] "RemoveContainer" containerID="c156c8304d0c3dbd1aa9ba6219858ee8ef45f9e5577b6664d501247eb48ce9a3"
Jul 29 15:49:23.955958 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:49:23.955943    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c156c8304d0c3dbd1aa9ba6219858ee8ef45f9e5577b6664d501247eb48ce9a3\": container with ID starting with c156c8304d0c3dbd1aa9ba6219858ee8ef45f9e5577b6664d501247eb48ce9a3 not found: ID does not exist" containerID="c156c8304d0c3dbd1aa9ba6219858ee8ef45f9e5577b6664d501247eb48ce9a3"
Jul 29 15:49:23.956007 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.955963    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c156c8304d0c3dbd1aa9ba6219858ee8ef45f9e5577b6664d501247eb48ce9a3"} err="failed to get container status \"c156c8304d0c3dbd1aa9ba6219858ee8ef45f9e5577b6664d501247eb48ce9a3\": rpc error: code = NotFound desc = could not find container \"c156c8304d0c3dbd1aa9ba6219858ee8ef45f9e5577b6664d501247eb48ce9a3\": container with ID starting with c156c8304d0c3dbd1aa9ba6219858ee8ef45f9e5577b6664d501247eb48ce9a3 not found: ID does not exist"
Jul 29 15:49:23.956007 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.955978    2560 scope.go:117] "RemoveContainer" containerID="3911c5bb645dacfba8f644e21908f8297616cdce549dc545c957db201b1fd2a4"
Jul 29 15:49:23.956252 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:49:23.956233    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3911c5bb645dacfba8f644e21908f8297616cdce549dc545c957db201b1fd2a4\": container with ID starting with 3911c5bb645dacfba8f644e21908f8297616cdce549dc545c957db201b1fd2a4 not found: ID does not exist" containerID="3911c5bb645dacfba8f644e21908f8297616cdce549dc545c957db201b1fd2a4"
Jul 29 15:49:23.956314 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.956256    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3911c5bb645dacfba8f644e21908f8297616cdce549dc545c957db201b1fd2a4"} err="failed to get container status \"3911c5bb645dacfba8f644e21908f8297616cdce549dc545c957db201b1fd2a4\": rpc error: code = NotFound desc = could not find container \"3911c5bb645dacfba8f644e21908f8297616cdce549dc545c957db201b1fd2a4\": container with ID starting with 3911c5bb645dacfba8f644e21908f8297616cdce549dc545c957db201b1fd2a4 not found: ID does not exist"
Jul 29 15:49:23.956314 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.956271    2560 scope.go:117] "RemoveContainer" containerID="37dfc12134c52829e36640edd25eb59972c807b8a19a13bb4bc72e7e436e5108"
Jul 29 15:49:23.956502 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:49:23.956486    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37dfc12134c52829e36640edd25eb59972c807b8a19a13bb4bc72e7e436e5108\": container with ID starting with 37dfc12134c52829e36640edd25eb59972c807b8a19a13bb4bc72e7e436e5108 not found: ID does not exist" containerID="37dfc12134c52829e36640edd25eb59972c807b8a19a13bb4bc72e7e436e5108"
Jul 29 15:49:23.956540 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.956507    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37dfc12134c52829e36640edd25eb59972c807b8a19a13bb4bc72e7e436e5108"} err="failed to get container status \"37dfc12134c52829e36640edd25eb59972c807b8a19a13bb4bc72e7e436e5108\": rpc error: code = NotFound desc = could not find container \"37dfc12134c52829e36640edd25eb59972c807b8a19a13bb4bc72e7e436e5108\": container with ID starting with 37dfc12134c52829e36640edd25eb59972c807b8a19a13bb4bc72e7e436e5108 not found: ID does not exist"
Jul 29 15:49:23.956540 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.956521    2560 scope.go:117] "RemoveContainer" containerID="e6893c098c7ecc010fe05c1ea8312845e28f9a2100a0ee4624dd3559ecdf26c3"
Jul 29 15:49:23.956756 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:49:23.956738    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6893c098c7ecc010fe05c1ea8312845e28f9a2100a0ee4624dd3559ecdf26c3\": container with ID starting with e6893c098c7ecc010fe05c1ea8312845e28f9a2100a0ee4624dd3559ecdf26c3 not found: ID does not exist" containerID="e6893c098c7ecc010fe05c1ea8312845e28f9a2100a0ee4624dd3559ecdf26c3"
Jul 29 15:49:23.956812 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.956761    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6893c098c7ecc010fe05c1ea8312845e28f9a2100a0ee4624dd3559ecdf26c3"} err="failed to get container status \"e6893c098c7ecc010fe05c1ea8312845e28f9a2100a0ee4624dd3559ecdf26c3\": rpc error: code = NotFound desc = could not find container \"e6893c098c7ecc010fe05c1ea8312845e28f9a2100a0ee4624dd3559ecdf26c3\": container with ID starting with e6893c098c7ecc010fe05c1ea8312845e28f9a2100a0ee4624dd3559ecdf26c3 not found: ID does not exist"
Jul 29 15:49:23.956812 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.956774    2560 scope.go:117] "RemoveContainer" containerID="273fe5de50af971c5440eea8778638ae3691a881f1984ded917f1498787a8e8e"
Jul 29 15:49:23.956999 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:49:23.956982    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"273fe5de50af971c5440eea8778638ae3691a881f1984ded917f1498787a8e8e\": container with ID starting with 273fe5de50af971c5440eea8778638ae3691a881f1984ded917f1498787a8e8e not found: ID does not exist" containerID="273fe5de50af971c5440eea8778638ae3691a881f1984ded917f1498787a8e8e"
Jul 29 15:49:23.957051 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.957002    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"273fe5de50af971c5440eea8778638ae3691a881f1984ded917f1498787a8e8e"} err="failed to get container status \"273fe5de50af971c5440eea8778638ae3691a881f1984ded917f1498787a8e8e\": rpc error: code = NotFound desc = could not find container \"273fe5de50af971c5440eea8778638ae3691a881f1984ded917f1498787a8e8e\": container with ID starting with 273fe5de50af971c5440eea8778638ae3691a881f1984ded917f1498787a8e8e not found: ID does not exist"
Jul 29 15:49:23.957051 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.957015    2560 scope.go:117] "RemoveContainer" containerID="c26d3981be2a2812d2f9df7e6135a54b81403c02f8254761cdaf78e818b8f516"
Jul 29 15:49:23.957221 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:49:23.957206    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c26d3981be2a2812d2f9df7e6135a54b81403c02f8254761cdaf78e818b8f516\": container with ID starting with c26d3981be2a2812d2f9df7e6135a54b81403c02f8254761cdaf78e818b8f516 not found: ID does not exist" containerID="c26d3981be2a2812d2f9df7e6135a54b81403c02f8254761cdaf78e818b8f516"
Jul 29 15:49:23.957263 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:23.957224    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26d3981be2a2812d2f9df7e6135a54b81403c02f8254761cdaf78e818b8f516"} err="failed to get container status \"c26d3981be2a2812d2f9df7e6135a54b81403c02f8254761cdaf78e818b8f516\": rpc error: code = NotFound desc = could not find container \"c26d3981be2a2812d2f9df7e6135a54b81403c02f8254761cdaf78e818b8f516\": container with ID starting with c26d3981be2a2812d2f9df7e6135a54b81403c02f8254761cdaf78e818b8f516 not found: ID does not exist"
Jul 29 15:49:24.000678 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.000656    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddf6857a-337e-46fb-864e-4458054c3387-metrics-client-ca\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:49:24.000678 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.000676    2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ddf6857a-337e-46fb-864e-4458054c3387-tls-assets\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:49:24.000804 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.000687    2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:49:24.000804 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.000695    2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ddf6857a-337e-46fb-864e-4458054c3387-config-out\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:49:24.000804 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.000702    2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-web-config\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:49:24.000804 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.000710    2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-cluster-tls-config\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:49:24.000804 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.000718    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-main-tls\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:49:24.000804 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.000728    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:49:24.000804 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.000736    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:49:24.000804 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.000745    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ddf6857a-337e-46fb-864e-4458054c3387-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:49:24.000804 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.000756    2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf6857a-337e-46fb-864e-4458054c3387-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:49:24.000804 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.000764    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrfmt\" (UniqueName: \"kubernetes.io/projected/ddf6857a-337e-46fb-864e-4458054c3387-kube-api-access-wrfmt\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:49:24.000804 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.000772    2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ddf6857a-337e-46fb-864e-4458054c3387-alertmanager-main-db\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:49:24.217387 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.217315    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:49:24.221901 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.221875    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:49:24.246538 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246513    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:49:24.246772 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246760    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="alertmanager"
Jul 29 15:49:24.246817 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246774    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="alertmanager"
Jul 29 15:49:24.246817 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246783    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dd620e8-a4e4-45e4-be96-0bc62db9a87b" containerName="registry"
Jul 29 15:49:24.246817 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246788    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dd620e8-a4e4-45e4-be96-0bc62db9a87b" containerName="registry"
Jul 29 15:49:24.246817 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246794    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="init-config-reloader"
Jul 29 15:49:24.246817 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246799    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="init-config-reloader"
Jul 29 15:49:24.246817 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246813    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="kube-rbac-proxy-web"
Jul 29 15:49:24.246817 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246818    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="kube-rbac-proxy-web"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246827    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="kube-rbac-proxy-metric"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246833    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="kube-rbac-proxy-metric"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246839    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="prom-label-proxy"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246844    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="prom-label-proxy"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246853    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="config-reloader"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246858    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="config-reloader"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246865    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="kube-rbac-proxy"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246870    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="kube-rbac-proxy"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246926    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="alertmanager"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246933    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="config-reloader"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246939    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="kube-rbac-proxy"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246946    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dd620e8-a4e4-45e4-be96-0bc62db9a87b" containerName="registry"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246952    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="kube-rbac-proxy-web"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246958    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="kube-rbac-proxy-metric"
Jul 29 15:49:24.247026 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.246963    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddf6857a-337e-46fb-864e-4458054c3387" containerName="prom-label-proxy"
Jul 29 15:49:24.250254 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.250239    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.253100 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.253083    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 15:49:24.253216 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.253104    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-85nxw\""
Jul 29 15:49:24.253328 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.253314    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 15:49:24.253392 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.253334    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 15:49:24.253494 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.253477    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 15:49:24.253552 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.253496    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 15:49:24.253552 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.253479    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 15:49:24.253552 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.253521    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 15:49:24.253695 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.253594    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 15:49:24.258173 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.258149    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 15:49:24.265763 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.265744    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:49:24.303749 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.303728    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.303851 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.303764    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/521c5283-7731-46ee-aa52-0e7057d2d5bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.303851 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.303784    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/521c5283-7731-46ee-aa52-0e7057d2d5bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.303851 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.303822    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl8q4\" (UniqueName: \"kubernetes.io/projected/521c5283-7731-46ee-aa52-0e7057d2d5bb-kube-api-access-kl8q4\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.303851 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.303843    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.304037 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.303878    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-web-config\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.304037 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.303906    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.304037 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.303943    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/521c5283-7731-46ee-aa52-0e7057d2d5bb-config-out\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.304037 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.303957    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.304037 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.303990    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/521c5283-7731-46ee-aa52-0e7057d2d5bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.304037 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.304011    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/521c5283-7731-46ee-aa52-0e7057d2d5bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.304037 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.304027    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.304265 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.304045    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.404753 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.404720    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/521c5283-7731-46ee-aa52-0e7057d2d5bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.404899 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.404802    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl8q4\" (UniqueName: \"kubernetes.io/projected/521c5283-7731-46ee-aa52-0e7057d2d5bb-kube-api-access-kl8q4\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.404899 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.404824    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.404899 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.404862    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-web-config\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.404899 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.404896    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.405142 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.404934    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/521c5283-7731-46ee-aa52-0e7057d2d5bb-config-out\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.405142 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.404956    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.405142 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.404993    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/521c5283-7731-46ee-aa52-0e7057d2d5bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.405142 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.405018    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/521c5283-7731-46ee-aa52-0e7057d2d5bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.405142 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.405040    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.405142 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.405070    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.405142 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.405099    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.405416 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.405145    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/521c5283-7731-46ee-aa52-0e7057d2d5bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.405969 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.405608    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/521c5283-7731-46ee-aa52-0e7057d2d5bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.405969 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.405789    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/521c5283-7731-46ee-aa52-0e7057d2d5bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.408445 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.407908    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/521c5283-7731-46ee-aa52-0e7057d2d5bb-config-out\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.408445 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.407965    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-web-config\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.408445 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.408028    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.408445 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.408174    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.408445 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.408336    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/521c5283-7731-46ee-aa52-0e7057d2d5bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.408445 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.408406    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.408844 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.408456    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/521c5283-7731-46ee-aa52-0e7057d2d5bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.408844 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.408510    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.408988 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.408969    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.409585 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.409569    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/521c5283-7731-46ee-aa52-0e7057d2d5bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.413519 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.413501    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl8q4\" (UniqueName: \"kubernetes.io/projected/521c5283-7731-46ee-aa52-0e7057d2d5bb-kube-api-access-kl8q4\") pod \"alertmanager-main-0\" (UID: \"521c5283-7731-46ee-aa52-0e7057d2d5bb\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.559532 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.559441    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:49:24.696361 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.696338    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:49:24.698629 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:49:24.698602    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod521c5283_7731_46ee_aa52_0e7057d2d5bb.slice/crio-a9363485add6d7a2142b6ac04da82e0950a7eb0679bfcbe38265d89df99c08b0 WatchSource:0}: Error finding container a9363485add6d7a2142b6ac04da82e0950a7eb0679bfcbe38265d89df99c08b0: Status 404 returned error can't find the container with id a9363485add6d7a2142b6ac04da82e0950a7eb0679bfcbe38265d89df99c08b0
Jul 29 15:49:24.903202 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.903168    2560 generic.go:356] "Generic (PLEG): container finished" podID="521c5283-7731-46ee-aa52-0e7057d2d5bb" containerID="57703e1671ca5a99816325e674325818a6922504a90d8d5f4337f413b528e4d2" exitCode=0
Jul 29 15:49:24.903583 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.903239    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"521c5283-7731-46ee-aa52-0e7057d2d5bb","Type":"ContainerDied","Data":"57703e1671ca5a99816325e674325818a6922504a90d8d5f4337f413b528e4d2"}
Jul 29 15:49:24.903583 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:24.903288    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"521c5283-7731-46ee-aa52-0e7057d2d5bb","Type":"ContainerStarted","Data":"a9363485add6d7a2142b6ac04da82e0950a7eb0679bfcbe38265d89df99c08b0"}
Jul 29 15:49:25.913552 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:25.913518    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"521c5283-7731-46ee-aa52-0e7057d2d5bb","Type":"ContainerStarted","Data":"7b2b04855d67713edd37844a922fd91afde7b83ccdb4166d2d6f865130cd39d6"}
Jul 29 15:49:25.913977 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:25.913608    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"521c5283-7731-46ee-aa52-0e7057d2d5bb","Type":"ContainerStarted","Data":"0d611b4905589d5c36446ec6aa122d32b2145016e7a505d2ece44cad2f77fb81"}
Jul 29 15:49:25.913977 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:25.913636    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"521c5283-7731-46ee-aa52-0e7057d2d5bb","Type":"ContainerStarted","Data":"2d233007d1c7f84752c0d9285c5359d1352f639790b1d3dca7243a8dc93cc406"}
Jul 29 15:49:25.913977 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:25.913660    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"521c5283-7731-46ee-aa52-0e7057d2d5bb","Type":"ContainerStarted","Data":"64a85487c836d44ac592d3970b8b200a7dd19a1fe62ce747b789b61d54500c70"}
Jul 29 15:49:25.913977 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:25.913686    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"521c5283-7731-46ee-aa52-0e7057d2d5bb","Type":"ContainerStarted","Data":"55a06dd7ad377028ec3e518b55d587ae41f0595c8a58b1ad1851dee9f33cf7c3"}
Jul 29 15:49:25.913977 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:25.913711    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"521c5283-7731-46ee-aa52-0e7057d2d5bb","Type":"ContainerStarted","Data":"fbd508bb77fc8db3247b7c6f667c227d942a919c98c5080a050fa14dfce7888c"}
Jul 29 15:49:25.943388 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:25.943222    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.927784041 podStartE2EDuration="1.943201937s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.417896ms" isStatefulPod=false podCreationTimestamp="2026-07-29 15:49:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:49:25.942377628 +0000 UTC m=+170.384583910" watchObservedRunningTime="2026-07-29 15:49:25.943201937 +0000 UTC m=+170.385408197"
Jul 29 15:49:26.042526 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:49:26.042494    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddf6857a-337e-46fb-864e-4458054c3387" path="/var/lib/kubelet/pods/ddf6857a-337e-46fb-864e-4458054c3387/volumes"
Jul 29 15:50:50.440514 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:50.440478    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jq8q4"]
Jul 29 15:50:50.442684 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:50.442662    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jq8q4"
Jul 29 15:50:50.442892 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:50.442866    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69b1807c-a31a-4082-ba98-0008b4aa3255-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jq8q4\" (UID: \"69b1807c-a31a-4082-ba98-0008b4aa3255\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jq8q4"
Jul 29 15:50:50.443035 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:50.442934    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v26m\" (UniqueName: \"kubernetes.io/projected/69b1807c-a31a-4082-ba98-0008b4aa3255-kube-api-access-2v26m\") pod \"cert-manager-operator-controller-manager-55c444bf86-jq8q4\" (UID: \"69b1807c-a31a-4082-ba98-0008b4aa3255\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jq8q4"
Jul 29 15:50:50.445752 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:50.445736    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-28bpg\""
Jul 29 15:50:50.445891 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:50.445861    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 29 15:50:50.446179 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:50.446160    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 29 15:50:50.459637 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:50.459617    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jq8q4"]
Jul 29 15:50:50.543825 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:50.543790    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69b1807c-a31a-4082-ba98-0008b4aa3255-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jq8q4\" (UID: \"69b1807c-a31a-4082-ba98-0008b4aa3255\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jq8q4"
Jul 29 15:50:50.543825 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:50.543828    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v26m\" (UniqueName: \"kubernetes.io/projected/69b1807c-a31a-4082-ba98-0008b4aa3255-kube-api-access-2v26m\") pod \"cert-manager-operator-controller-manager-55c444bf86-jq8q4\" (UID: \"69b1807c-a31a-4082-ba98-0008b4aa3255\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jq8q4"
Jul 29 15:50:50.544188 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:50.544169    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69b1807c-a31a-4082-ba98-0008b4aa3255-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jq8q4\" (UID: \"69b1807c-a31a-4082-ba98-0008b4aa3255\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jq8q4"
Jul 29 15:50:50.552877 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:50.552854    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v26m\" (UniqueName: \"kubernetes.io/projected/69b1807c-a31a-4082-ba98-0008b4aa3255-kube-api-access-2v26m\") pod \"cert-manager-operator-controller-manager-55c444bf86-jq8q4\" (UID: \"69b1807c-a31a-4082-ba98-0008b4aa3255\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jq8q4"
Jul 29 15:50:50.752127 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:50.752047    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jq8q4"
Jul 29 15:50:50.878201 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:50.878177    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jq8q4"]
Jul 29 15:50:50.880850 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:50:50.880817    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69b1807c_a31a_4082_ba98_0008b4aa3255.slice/crio-5915aeb4caafaa3d418ae9b6dee96a2d63855abf8c0c84c2ae8350a542365bd0 WatchSource:0}: Error finding container 5915aeb4caafaa3d418ae9b6dee96a2d63855abf8c0c84c2ae8350a542365bd0: Status 404 returned error can't find the container with id 5915aeb4caafaa3d418ae9b6dee96a2d63855abf8c0c84c2ae8350a542365bd0
Jul 29 15:50:51.207970 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:51.207938    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jq8q4" event={"ID":"69b1807c-a31a-4082-ba98-0008b4aa3255","Type":"ContainerStarted","Data":"5915aeb4caafaa3d418ae9b6dee96a2d63855abf8c0c84c2ae8350a542365bd0"}
Jul 29 15:50:54.222348 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:54.222314    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jq8q4" event={"ID":"69b1807c-a31a-4082-ba98-0008b4aa3255","Type":"ContainerStarted","Data":"d5a316456677dc36d4136baf43c7fecd0b2fb9cdd2a6b3ff8a3cf4f7c420bcc2"}
Jul 29 15:50:54.247420 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:54.247140    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jq8q4" podStartSLOduration=1.924255691 podStartE2EDuration="4.247125166s" totalImagesPullingTime="2.322869475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:50:54.245501089 +0000 UTC m=+258.687707358" watchObservedRunningTime="2026-07-29 15:50:54.247125166 +0000 UTC m=+258.689331425"
Jul 29 15:50:54.943668 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:54.943630    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"]
Jul 29 15:50:54.946072 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:54.946051    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"
Jul 29 15:50:54.949645 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:54.949624    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 15:50:54.949645 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:54.949641    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9vjkx\""
Jul 29 15:50:54.949796 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:54.949762    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 15:50:54.961036 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:54.961017    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"]
Jul 29 15:50:54.978677 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:54.978645    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bddc4d3c-344f-425e-b277-35f7d4bca5af-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w\" (UID: \"bddc4d3c-344f-425e-b277-35f7d4bca5af\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"
Jul 29 15:50:54.978776 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:54.978693    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bddc4d3c-344f-425e-b277-35f7d4bca5af-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w\" (UID: \"bddc4d3c-344f-425e-b277-35f7d4bca5af\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"
Jul 29 15:50:54.978776 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:54.978740    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thd6r\" (UniqueName: \"kubernetes.io/projected/bddc4d3c-344f-425e-b277-35f7d4bca5af-kube-api-access-thd6r\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w\" (UID: \"bddc4d3c-344f-425e-b277-35f7d4bca5af\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"
Jul 29 15:50:55.079114 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:55.079083    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bddc4d3c-344f-425e-b277-35f7d4bca5af-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w\" (UID: \"bddc4d3c-344f-425e-b277-35f7d4bca5af\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"
Jul 29 15:50:55.079114 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:55.079114    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bddc4d3c-344f-425e-b277-35f7d4bca5af-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w\" (UID: \"bddc4d3c-344f-425e-b277-35f7d4bca5af\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"
Jul 29 15:50:55.079307 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:55.079159    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thd6r\" (UniqueName: \"kubernetes.io/projected/bddc4d3c-344f-425e-b277-35f7d4bca5af-kube-api-access-thd6r\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w\" (UID: \"bddc4d3c-344f-425e-b277-35f7d4bca5af\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"
Jul 29 15:50:55.079517 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:55.079498    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bddc4d3c-344f-425e-b277-35f7d4bca5af-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w\" (UID: \"bddc4d3c-344f-425e-b277-35f7d4bca5af\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"
Jul 29 15:50:55.079558 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:55.079507    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bddc4d3c-344f-425e-b277-35f7d4bca5af-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w\" (UID: \"bddc4d3c-344f-425e-b277-35f7d4bca5af\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"
Jul 29 15:50:55.090686 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:55.090657    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thd6r\" (UniqueName: \"kubernetes.io/projected/bddc4d3c-344f-425e-b277-35f7d4bca5af-kube-api-access-thd6r\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w\" (UID: \"bddc4d3c-344f-425e-b277-35f7d4bca5af\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"
Jul 29 15:50:55.254791 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:55.254714    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"
Jul 29 15:50:55.382540 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:55.382387    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"]
Jul 29 15:50:55.385035 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:50:55.385000    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbddc4d3c_344f_425e_b277_35f7d4bca5af.slice/crio-ebb5ce92c560e8506ca56fee815d865708ec6c1f5aa31c6b684e0f8d8cd1e867 WatchSource:0}: Error finding container ebb5ce92c560e8506ca56fee815d865708ec6c1f5aa31c6b684e0f8d8cd1e867: Status 404 returned error can't find the container with id ebb5ce92c560e8506ca56fee815d865708ec6c1f5aa31c6b684e0f8d8cd1e867
Jul 29 15:50:56.232410 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:50:56.232371    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w" event={"ID":"bddc4d3c-344f-425e-b277-35f7d4bca5af","Type":"ContainerStarted","Data":"ebb5ce92c560e8506ca56fee815d865708ec6c1f5aa31c6b684e0f8d8cd1e867"}
Jul 29 15:51:01.475328 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:01.475294    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-nswtp"]
Jul 29 15:51:01.477319 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:01.477277    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nswtp"
Jul 29 15:51:01.480290 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:01.480268    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 15:51:01.480290 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:01.480278    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 15:51:01.480290 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:01.480277    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8c527\""
Jul 29 15:51:01.487838 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:01.487819    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-nswtp"]
Jul 29 15:51:01.533301 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:01.533272    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whrzj\" (UniqueName: \"kubernetes.io/projected/8573f7a7-ff0b-4dc8-a4b3-e749bea3bf8a-kube-api-access-whrzj\") pod \"cert-manager-cainjector-559cdd8c87-nswtp\" (UID: \"8573f7a7-ff0b-4dc8-a4b3-e749bea3bf8a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nswtp"
Jul 29 15:51:01.533452 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:01.533307    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8573f7a7-ff0b-4dc8-a4b3-e749bea3bf8a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-nswtp\" (UID: \"8573f7a7-ff0b-4dc8-a4b3-e749bea3bf8a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nswtp"
Jul 29 15:51:01.634291 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:01.634258    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whrzj\" (UniqueName: \"kubernetes.io/projected/8573f7a7-ff0b-4dc8-a4b3-e749bea3bf8a-kube-api-access-whrzj\") pod \"cert-manager-cainjector-559cdd8c87-nswtp\" (UID: \"8573f7a7-ff0b-4dc8-a4b3-e749bea3bf8a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nswtp"
Jul 29 15:51:01.634291 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:01.634291    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8573f7a7-ff0b-4dc8-a4b3-e749bea3bf8a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-nswtp\" (UID: \"8573f7a7-ff0b-4dc8-a4b3-e749bea3bf8a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nswtp"
Jul 29 15:51:01.643527 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:01.643490    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8573f7a7-ff0b-4dc8-a4b3-e749bea3bf8a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-nswtp\" (UID: \"8573f7a7-ff0b-4dc8-a4b3-e749bea3bf8a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nswtp"
Jul 29 15:51:01.643686 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:01.643668    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whrzj\" (UniqueName: \"kubernetes.io/projected/8573f7a7-ff0b-4dc8-a4b3-e749bea3bf8a-kube-api-access-whrzj\") pod \"cert-manager-cainjector-559cdd8c87-nswtp\" (UID: \"8573f7a7-ff0b-4dc8-a4b3-e749bea3bf8a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nswtp"
Jul 29 15:51:01.785732 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:01.785631    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nswtp"
Jul 29 15:51:01.912745 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:01.912715    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-nswtp"]
Jul 29 15:51:01.915679 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:51:01.915650    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8573f7a7_ff0b_4dc8_a4b3_e749bea3bf8a.slice/crio-7465de73428332e7020cfd0802205e84e5df092ba3aac51c25fac5155cabc0d2 WatchSource:0}: Error finding container 7465de73428332e7020cfd0802205e84e5df092ba3aac51c25fac5155cabc0d2: Status 404 returned error can't find the container with id 7465de73428332e7020cfd0802205e84e5df092ba3aac51c25fac5155cabc0d2
Jul 29 15:51:02.260487 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:02.260450    2560 generic.go:356] "Generic (PLEG): container finished" podID="bddc4d3c-344f-425e-b277-35f7d4bca5af" containerID="9344356e6f6557de91eb608649a044750c76489ea4eb0a0c7122ccf825dbd11c" exitCode=0
Jul 29 15:51:02.260681 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:02.260503    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w" event={"ID":"bddc4d3c-344f-425e-b277-35f7d4bca5af","Type":"ContainerDied","Data":"9344356e6f6557de91eb608649a044750c76489ea4eb0a0c7122ccf825dbd11c"}
Jul 29 15:51:02.261703 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:02.261678    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nswtp" event={"ID":"8573f7a7-ff0b-4dc8-a4b3-e749bea3bf8a","Type":"ContainerStarted","Data":"7465de73428332e7020cfd0802205e84e5df092ba3aac51c25fac5155cabc0d2"}
Jul 29 15:51:05.277687 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:05.277651    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nswtp" event={"ID":"8573f7a7-ff0b-4dc8-a4b3-e749bea3bf8a","Type":"ContainerStarted","Data":"a29f2712d85fc9342c1df10b9a4c18b5bfbbf6d934ce8f9fbc0666c5c200cb86"}
Jul 29 15:51:20.336271 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:20.336183    2560 generic.go:356] "Generic (PLEG): container finished" podID="bddc4d3c-344f-425e-b277-35f7d4bca5af" containerID="bcdbccd6118de31aff05b5cbc4fb1ffa04d282ae61939f64645e6be3b8828253" exitCode=0
Jul 29 15:51:20.336271 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:20.336227    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w" event={"ID":"bddc4d3c-344f-425e-b277-35f7d4bca5af","Type":"ContainerDied","Data":"bcdbccd6118de31aff05b5cbc4fb1ffa04d282ae61939f64645e6be3b8828253"}
Jul 29 15:51:20.359383 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:20.359335    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nswtp" podStartSLOduration=16.846749196 podStartE2EDuration="19.359320784s" totalImagesPullingTime="2.512571588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:51:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:51:05.305887154 +0000 UTC m=+269.748093412" watchObservedRunningTime="2026-07-29 15:51:20.359320784 +0000 UTC m=+284.801527043"
Jul 29 15:51:28.374406 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:28.374370    2560 generic.go:356] "Generic (PLEG): container finished" podID="bddc4d3c-344f-425e-b277-35f7d4bca5af" containerID="96f64c8c137e6845aa4d12c3bc604c88114ed5d8fbfde6ffb083ff2b180abea6" exitCode=0
Jul 29 15:51:28.374786 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:28.374447    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w" event={"ID":"bddc4d3c-344f-425e-b277-35f7d4bca5af","Type":"ContainerDied","Data":"96f64c8c137e6845aa4d12c3bc604c88114ed5d8fbfde6ffb083ff2b180abea6"}
Jul 29 15:51:29.503432 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:29.503410    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"
Jul 29 15:51:29.673326 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:29.673299    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bddc4d3c-344f-425e-b277-35f7d4bca5af-util\") pod \"bddc4d3c-344f-425e-b277-35f7d4bca5af\" (UID: \"bddc4d3c-344f-425e-b277-35f7d4bca5af\") "
Jul 29 15:51:29.673515 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:29.673345    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thd6r\" (UniqueName: \"kubernetes.io/projected/bddc4d3c-344f-425e-b277-35f7d4bca5af-kube-api-access-thd6r\") pod \"bddc4d3c-344f-425e-b277-35f7d4bca5af\" (UID: \"bddc4d3c-344f-425e-b277-35f7d4bca5af\") "
Jul 29 15:51:29.673515 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:29.673414    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bddc4d3c-344f-425e-b277-35f7d4bca5af-bundle\") pod \"bddc4d3c-344f-425e-b277-35f7d4bca5af\" (UID: \"bddc4d3c-344f-425e-b277-35f7d4bca5af\") "
Jul 29 15:51:29.673821 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:29.673799    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bddc4d3c-344f-425e-b277-35f7d4bca5af-bundle" (OuterVolumeSpecName: "bundle") pod "bddc4d3c-344f-425e-b277-35f7d4bca5af" (UID: "bddc4d3c-344f-425e-b277-35f7d4bca5af"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:51:29.675528 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:29.675506    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bddc4d3c-344f-425e-b277-35f7d4bca5af-kube-api-access-thd6r" (OuterVolumeSpecName: "kube-api-access-thd6r") pod "bddc4d3c-344f-425e-b277-35f7d4bca5af" (UID: "bddc4d3c-344f-425e-b277-35f7d4bca5af"). InnerVolumeSpecName "kube-api-access-thd6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:51:29.678071 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:29.678046    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bddc4d3c-344f-425e-b277-35f7d4bca5af-util" (OuterVolumeSpecName: "util") pod "bddc4d3c-344f-425e-b277-35f7d4bca5af" (UID: "bddc4d3c-344f-425e-b277-35f7d4bca5af"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:51:29.774754 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:29.774712    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bddc4d3c-344f-425e-b277-35f7d4bca5af-bundle\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:51:29.774754 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:29.774748    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bddc4d3c-344f-425e-b277-35f7d4bca5af-util\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:51:29.774754 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:29.774758    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thd6r\" (UniqueName: \"kubernetes.io/projected/bddc4d3c-344f-425e-b277-35f7d4bca5af-kube-api-access-thd6r\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:51:30.385361 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:30.385271    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w" event={"ID":"bddc4d3c-344f-425e-b277-35f7d4bca5af","Type":"ContainerDied","Data":"ebb5ce92c560e8506ca56fee815d865708ec6c1f5aa31c6b684e0f8d8cd1e867"}
Jul 29 15:51:30.385361 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:30.385304    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd7f7w"
Jul 29 15:51:30.385361 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:30.385319    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebb5ce92c560e8506ca56fee815d865708ec6c1f5aa31c6b684e0f8d8cd1e867"
Jul 29 15:51:33.859555 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:33.859525    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hxgdj"]
Jul 29 15:51:33.859957 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:33.859799    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bddc4d3c-344f-425e-b277-35f7d4bca5af" containerName="pull"
Jul 29 15:51:33.859957 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:33.859810    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bddc4d3c-344f-425e-b277-35f7d4bca5af" containerName="pull"
Jul 29 15:51:33.859957 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:33.859828    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bddc4d3c-344f-425e-b277-35f7d4bca5af" containerName="util"
Jul 29 15:51:33.859957 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:33.859833    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bddc4d3c-344f-425e-b277-35f7d4bca5af" containerName="util"
Jul 29 15:51:33.859957 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:33.859840    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bddc4d3c-344f-425e-b277-35f7d4bca5af" containerName="extract"
Jul 29 15:51:33.859957 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:33.859846    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bddc4d3c-344f-425e-b277-35f7d4bca5af" containerName="extract"
Jul 29 15:51:33.859957 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:33.859901    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bddc4d3c-344f-425e-b277-35f7d4bca5af" containerName="extract"
Jul 29 15:51:33.862750 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:33.862735    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hxgdj"
Jul 29 15:51:33.866004 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:33.865984    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 15:51:33.867261 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:33.867243    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 15:51:33.867349 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:33.867257    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-sq2x5\""
Jul 29 15:51:33.871839 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:33.871818    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hxgdj"]
Jul 29 15:51:34.008597 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:34.008552    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hxgdj\" (UID: \"7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hxgdj"
Jul 29 15:51:34.008597 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:34.008597    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsx4v\" (UniqueName: \"kubernetes.io/projected/7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15-kube-api-access-gsx4v\") pod \"openshift-lws-operator-fd8ccff4c-hxgdj\" (UID: \"7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hxgdj"
Jul 29 15:51:34.109439 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:34.109408    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hxgdj\" (UID: \"7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hxgdj"
Jul 29 15:51:34.109439 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:34.109444    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsx4v\" (UniqueName: \"kubernetes.io/projected/7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15-kube-api-access-gsx4v\") pod \"openshift-lws-operator-fd8ccff4c-hxgdj\" (UID: \"7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hxgdj"
Jul 29 15:51:34.109820 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:34.109777    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hxgdj\" (UID: \"7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hxgdj"
Jul 29 15:51:34.119487 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:34.119464    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsx4v\" (UniqueName: \"kubernetes.io/projected/7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15-kube-api-access-gsx4v\") pod \"openshift-lws-operator-fd8ccff4c-hxgdj\" (UID: \"7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hxgdj"
Jul 29 15:51:34.171949 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:34.171904    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hxgdj"
Jul 29 15:51:34.303649 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:34.303489    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hxgdj"]
Jul 29 15:51:34.306159 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:51:34.306133    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a49ed7a_ce4e_4ef6_9a89_86e6eaac5c15.slice/crio-d99b7459b8665c08d904ad63fa46d18a1153431b3f93aa69260c47c1cbc6c863 WatchSource:0}: Error finding container d99b7459b8665c08d904ad63fa46d18a1153431b3f93aa69260c47c1cbc6c863: Status 404 returned error can't find the container with id d99b7459b8665c08d904ad63fa46d18a1153431b3f93aa69260c47c1cbc6c863
Jul 29 15:51:34.403515 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:34.403489    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hxgdj" event={"ID":"7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15","Type":"ContainerStarted","Data":"d99b7459b8665c08d904ad63fa46d18a1153431b3f93aa69260c47c1cbc6c863"}
Jul 29 15:51:35.873509 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:35.873477    2560 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 15:51:37.418005 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:37.417968    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hxgdj" event={"ID":"7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15","Type":"ContainerStarted","Data":"40f71af08c3f66f7309548f68cd6d5ff9584e7be7094e31864d0e670c32b4d4b"}
Jul 29 15:51:37.438509 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:37.438460    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hxgdj" podStartSLOduration=2.357443975 podStartE2EDuration="4.438441268s" totalImagesPullingTime="2.080997293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:51:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:51:37.437863257 +0000 UTC m=+301.880069516" watchObservedRunningTime="2026-07-29 15:51:37.438441268 +0000 UTC m=+301.880647527"
Jul 29 15:51:44.725770 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.725734    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"]
Jul 29 15:51:44.729098 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.729080    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:44.734393 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.734366    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8psf9\""
Jul 29 15:51:44.734495 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.734397    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 29 15:51:44.734495 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.734366    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 29 15:51:44.734495 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.734442    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 29 15:51:44.748624 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.748592    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"]
Jul 29 15:51:44.796395 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.796370    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0516333f-fe64-4021-b4c3-fbbbfd57c85b-metrics-cert\") pod \"lws-controller-manager-7cb94b984b-c8gn9\" (UID: \"0516333f-fe64-4021-b4c3-fbbbfd57c85b\") " pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:44.796507 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.796414    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0516333f-fe64-4021-b4c3-fbbbfd57c85b-manager-config\") pod \"lws-controller-manager-7cb94b984b-c8gn9\" (UID: \"0516333f-fe64-4021-b4c3-fbbbfd57c85b\") " pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:44.796507 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.796431    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0516333f-fe64-4021-b4c3-fbbbfd57c85b-cert\") pod \"lws-controller-manager-7cb94b984b-c8gn9\" (UID: \"0516333f-fe64-4021-b4c3-fbbbfd57c85b\") " pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:44.796580 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.796526    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znssf\" (UniqueName: \"kubernetes.io/projected/0516333f-fe64-4021-b4c3-fbbbfd57c85b-kube-api-access-znssf\") pod \"lws-controller-manager-7cb94b984b-c8gn9\" (UID: \"0516333f-fe64-4021-b4c3-fbbbfd57c85b\") " pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:44.897252 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.897219    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znssf\" (UniqueName: \"kubernetes.io/projected/0516333f-fe64-4021-b4c3-fbbbfd57c85b-kube-api-access-znssf\") pod \"lws-controller-manager-7cb94b984b-c8gn9\" (UID: \"0516333f-fe64-4021-b4c3-fbbbfd57c85b\") " pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:44.897458 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.897261    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0516333f-fe64-4021-b4c3-fbbbfd57c85b-metrics-cert\") pod \"lws-controller-manager-7cb94b984b-c8gn9\" (UID: \"0516333f-fe64-4021-b4c3-fbbbfd57c85b\") " pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:44.897458 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.897291    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0516333f-fe64-4021-b4c3-fbbbfd57c85b-manager-config\") pod \"lws-controller-manager-7cb94b984b-c8gn9\" (UID: \"0516333f-fe64-4021-b4c3-fbbbfd57c85b\") " pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:44.897458 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.897305    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0516333f-fe64-4021-b4c3-fbbbfd57c85b-cert\") pod \"lws-controller-manager-7cb94b984b-c8gn9\" (UID: \"0516333f-fe64-4021-b4c3-fbbbfd57c85b\") " pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:44.898038 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.898014    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0516333f-fe64-4021-b4c3-fbbbfd57c85b-manager-config\") pod \"lws-controller-manager-7cb94b984b-c8gn9\" (UID: \"0516333f-fe64-4021-b4c3-fbbbfd57c85b\") " pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:44.899744 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.899720    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0516333f-fe64-4021-b4c3-fbbbfd57c85b-cert\") pod \"lws-controller-manager-7cb94b984b-c8gn9\" (UID: \"0516333f-fe64-4021-b4c3-fbbbfd57c85b\") " pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:44.899819 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.899782    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0516333f-fe64-4021-b4c3-fbbbfd57c85b-metrics-cert\") pod \"lws-controller-manager-7cb94b984b-c8gn9\" (UID: \"0516333f-fe64-4021-b4c3-fbbbfd57c85b\") " pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:44.906423 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:44.906402    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znssf\" (UniqueName: \"kubernetes.io/projected/0516333f-fe64-4021-b4c3-fbbbfd57c85b-kube-api-access-znssf\") pod \"lws-controller-manager-7cb94b984b-c8gn9\" (UID: \"0516333f-fe64-4021-b4c3-fbbbfd57c85b\") " pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:45.038131 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:45.038050    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:45.201266 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:45.201125    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"]
Jul 29 15:51:45.204050 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:51:45.204020    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0516333f_fe64_4021_b4c3_fbbbfd57c85b.slice/crio-5a7bb177a108617f62ad4a46bb6017aee8489ce8e810120c97fa99ec2ee35b32 WatchSource:0}: Error finding container 5a7bb177a108617f62ad4a46bb6017aee8489ce8e810120c97fa99ec2ee35b32: Status 404 returned error can't find the container with id 5a7bb177a108617f62ad4a46bb6017aee8489ce8e810120c97fa99ec2ee35b32
Jul 29 15:51:45.205855 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:45.205839    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 15:51:45.453908 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:45.453876    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9" event={"ID":"0516333f-fe64-4021-b4c3-fbbbfd57c85b","Type":"ContainerStarted","Data":"5a7bb177a108617f62ad4a46bb6017aee8489ce8e810120c97fa99ec2ee35b32"}
Jul 29 15:51:47.464590 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:47.464551    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9" event={"ID":"0516333f-fe64-4021-b4c3-fbbbfd57c85b","Type":"ContainerStarted","Data":"8d977a1654fcdbb1176a500a8cf143be4cc3c737e48b3dcf3a1eeb7a8c22ee56"}
Jul 29 15:51:47.465016 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:47.464608    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:47.484523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:47.484481    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9" podStartSLOduration=1.9138125289999999 podStartE2EDuration="3.48446638s" totalImagesPullingTime="1.570653851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:51:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:51:47.482159216 +0000 UTC m=+311.924365476" watchObservedRunningTime="2026-07-29 15:51:47.48446638 +0000 UTC m=+311.926672639"
Jul 29 15:51:54.965658 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:54.965624    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"]
Jul 29 15:51:54.971700 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:54.971675    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:54.975132 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:54.975103    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 15:51:54.975244 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:54.975164    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k69gf\""
Jul 29 15:51:54.975244 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:54.975168    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 15:51:54.976093 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:54.976078    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 15:51:54.979609 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:54.979592    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 15:51:55.001096 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.001078    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"]
Jul 29 15:51:55.068690 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.068659    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6w2x\" (UniqueName: \"kubernetes.io/projected/7238c72e-3821-425c-8c55-e7e65ac8d3da-kube-api-access-b6w2x\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.068690 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.068694    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7238c72e-3821-425c-8c55-e7e65ac8d3da-webhook-cert\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.068893 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.068723    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7238c72e-3821-425c-8c55-e7e65ac8d3da-manifests\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.068893 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.068786    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7238c72e-3821-425c-8c55-e7e65ac8d3da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.068893 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.068826    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7238c72e-3821-425c-8c55-e7e65ac8d3da-tmp\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.169564 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.169531    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7238c72e-3821-425c-8c55-e7e65ac8d3da-manifests\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.169750 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.169574    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7238c72e-3821-425c-8c55-e7e65ac8d3da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.169750 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.169601    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7238c72e-3821-425c-8c55-e7e65ac8d3da-tmp\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.169865 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.169746    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6w2x\" (UniqueName: \"kubernetes.io/projected/7238c72e-3821-425c-8c55-e7e65ac8d3da-kube-api-access-b6w2x\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.169865 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.169780    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7238c72e-3821-425c-8c55-e7e65ac8d3da-webhook-cert\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.170051 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.170032    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7238c72e-3821-425c-8c55-e7e65ac8d3da-manifests\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.172410 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.172381    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7238c72e-3821-425c-8c55-e7e65ac8d3da-tmp\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.172523 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.172505    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7238c72e-3821-425c-8c55-e7e65ac8d3da-webhook-cert\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.172718 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.172699    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7238c72e-3821-425c-8c55-e7e65ac8d3da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.189859 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.189830    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6w2x\" (UniqueName: \"kubernetes.io/projected/7238c72e-3821-425c-8c55-e7e65ac8d3da-kube-api-access-b6w2x\") pod \"opendatahub-operator-controller-manager-75b794959f-zwzb7\" (UID: \"7238c72e-3821-425c-8c55-e7e65ac8d3da\") " pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.281294 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.281210    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:51:55.434152 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.434058    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"]
Jul 29 15:51:55.436849 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:51:55.436823    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7238c72e_3821_425c_8c55_e7e65ac8d3da.slice/crio-e380e2ce447ecfe5117122390b32029f03c77e1161bbf3af3fe30cc7f200f4c8 WatchSource:0}: Error finding container e380e2ce447ecfe5117122390b32029f03c77e1161bbf3af3fe30cc7f200f4c8: Status 404 returned error can't find the container with id e380e2ce447ecfe5117122390b32029f03c77e1161bbf3af3fe30cc7f200f4c8
Jul 29 15:51:55.497714 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:55.497682    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7" event={"ID":"7238c72e-3821-425c-8c55-e7e65ac8d3da","Type":"ContainerStarted","Data":"e380e2ce447ecfe5117122390b32029f03c77e1161bbf3af3fe30cc7f200f4c8"}
Jul 29 15:51:58.471208 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:58.471186    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7cb94b984b-c8gn9"
Jul 29 15:51:58.515371 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:58.515336    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7" event={"ID":"7238c72e-3821-425c-8c55-e7e65ac8d3da","Type":"ContainerStarted","Data":"e3c1d580c894d8326946c94e1aaa3a55ea3fc08e65516983a0f6386074a9126a"}
Jul 29 15:51:59.521173 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:59.521136    2560 generic.go:356] "Generic (PLEG): container finished" podID="7238c72e-3821-425c-8c55-e7e65ac8d3da" containerID="e3c1d580c894d8326946c94e1aaa3a55ea3fc08e65516983a0f6386074a9126a" exitCode=0
Jul 29 15:51:59.521562 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:51:59.521220    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7" event={"ID":"7238c72e-3821-425c-8c55-e7e65ac8d3da","Type":"ContainerDied","Data":"e3c1d580c894d8326946c94e1aaa3a55ea3fc08e65516983a0f6386074a9126a"}
Jul 29 15:52:00.528045 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:00.528005    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7" event={"ID":"7238c72e-3821-425c-8c55-e7e65ac8d3da","Type":"ContainerStarted","Data":"f5b293c95e81e43db64e5b28c46ce34348bee139eead10badb21ae75e6c841d5"}
Jul 29 15:52:00.528514 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:00.528081    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:52:00.552813 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:00.552767    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7" podStartSLOduration=1.9600549919999999 podStartE2EDuration="6.552750529s" totalImagesPullingTime="3.539651163s" totalInitContainerRuntime="1.053044374s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:51:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:52:00.551086023 +0000 UTC m=+324.993292333" watchObservedRunningTime="2026-07-29 15:52:00.552750529 +0000 UTC m=+324.994956788"
Jul 29 15:52:11.534494 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:11.534463    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-75b794959f-zwzb7"
Jul 29 15:52:15.803793 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:15.803752    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q"]
Jul 29 15:52:15.808619 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:15.808600    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q"
Jul 29 15:52:15.812474 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:15.812453    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 15:52:15.812604 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:15.812458    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 15:52:15.813780 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:15.813760    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 15:52:15.813910 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:15.813871    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-lxtvq\""
Jul 29 15:52:15.814068 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:15.813999    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 15:52:15.821830 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:15.821808    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q"]
Jul 29 15:52:15.924284 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:15.924251    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97qmk\" (UniqueName: \"kubernetes.io/projected/2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf-kube-api-access-97qmk\") pod \"kube-auth-proxy-86bbb74d46-5qf7q\" (UID: \"2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf\") " pod="openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q"
Jul 29 15:52:15.924284 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:15.924285    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf-tls-certs\") pod \"kube-auth-proxy-86bbb74d46-5qf7q\" (UID: \"2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf\") " pod="openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q"
Jul 29 15:52:15.924498 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:15.924348    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf-tmp\") pod \"kube-auth-proxy-86bbb74d46-5qf7q\" (UID: \"2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf\") " pod="openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q"
Jul 29 15:52:16.025307 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:16.025265    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97qmk\" (UniqueName: \"kubernetes.io/projected/2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf-kube-api-access-97qmk\") pod \"kube-auth-proxy-86bbb74d46-5qf7q\" (UID: \"2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf\") " pod="openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q"
Jul 29 15:52:16.025307 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:16.025312    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf-tls-certs\") pod \"kube-auth-proxy-86bbb74d46-5qf7q\" (UID: \"2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf\") " pod="openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q"
Jul 29 15:52:16.025552 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:16.025381    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf-tmp\") pod \"kube-auth-proxy-86bbb74d46-5qf7q\" (UID: \"2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf\") " pod="openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q"
Jul 29 15:52:16.027636 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:16.027608    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf-tmp\") pod \"kube-auth-proxy-86bbb74d46-5qf7q\" (UID: \"2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf\") " pod="openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q"
Jul 29 15:52:16.027811 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:16.027794    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf-tls-certs\") pod \"kube-auth-proxy-86bbb74d46-5qf7q\" (UID: \"2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf\") " pod="openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q"
Jul 29 15:52:16.038551 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:16.038526    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97qmk\" (UniqueName: \"kubernetes.io/projected/2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf-kube-api-access-97qmk\") pod \"kube-auth-proxy-86bbb74d46-5qf7q\" (UID: \"2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf\") " pod="openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q"
Jul 29 15:52:16.119198 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:16.119117    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q"
Jul 29 15:52:16.258120 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:16.258095    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q"]
Jul 29 15:52:16.260362 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:52:16.260335    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f91b2f6_1173_4fcd_95df_e41e5dc4a1bf.slice/crio-76eca3d084db07066ed899e193a1769a17803705e0d4576d14d379085baa22bf WatchSource:0}: Error finding container 76eca3d084db07066ed899e193a1769a17803705e0d4576d14d379085baa22bf: Status 404 returned error can't find the container with id 76eca3d084db07066ed899e193a1769a17803705e0d4576d14d379085baa22bf
Jul 29 15:52:16.597016 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:16.596982    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q" event={"ID":"2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf","Type":"ContainerStarted","Data":"76eca3d084db07066ed899e193a1769a17803705e0d4576d14d379085baa22bf"}
Jul 29 15:52:19.613458 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:19.613419    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q" event={"ID":"2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf","Type":"ContainerStarted","Data":"fb720735aec685588130df804766c11559007faf70abc362168273a065d6eb26"}
Jul 29 15:52:19.634348 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:52:19.634297    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-86bbb74d46-5qf7q" podStartSLOduration=2.079946223 podStartE2EDuration="4.634282284s" totalImagesPullingTime="2.554336061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:52:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:52:19.632691108 +0000 UTC m=+344.074897379" watchObservedRunningTime="2026-07-29 15:52:19.634282284 +0000 UTC m=+344.076488554"
Jul 29 15:53:31.134636 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.134604    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"]
Jul 29 15:53:31.138016 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.138000    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"
Jul 29 15:53:31.141048 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.141029    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 15:53:31.142211 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.142192    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 15:53:31.142292 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.142196    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-52j2g\""
Jul 29 15:53:31.149385 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.149366    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"]
Jul 29 15:53:31.254872 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.254843    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/147ac01f-e082-4b0a-8113-789f6585b6c6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt\" (UID: \"147ac01f-e082-4b0a-8113-789f6585b6c6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"
Jul 29 15:53:31.255071 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.254884    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/147ac01f-e082-4b0a-8113-789f6585b6c6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt\" (UID: \"147ac01f-e082-4b0a-8113-789f6585b6c6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"
Jul 29 15:53:31.255071 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.255033    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8s5p\" (UniqueName: \"kubernetes.io/projected/147ac01f-e082-4b0a-8113-789f6585b6c6-kube-api-access-t8s5p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt\" (UID: \"147ac01f-e082-4b0a-8113-789f6585b6c6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"
Jul 29 15:53:31.356356 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.356325    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8s5p\" (UniqueName: \"kubernetes.io/projected/147ac01f-e082-4b0a-8113-789f6585b6c6-kube-api-access-t8s5p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt\" (UID: \"147ac01f-e082-4b0a-8113-789f6585b6c6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"
Jul 29 15:53:31.356532 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.356364    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/147ac01f-e082-4b0a-8113-789f6585b6c6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt\" (UID: \"147ac01f-e082-4b0a-8113-789f6585b6c6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"
Jul 29 15:53:31.356532 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.356386    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/147ac01f-e082-4b0a-8113-789f6585b6c6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt\" (UID: \"147ac01f-e082-4b0a-8113-789f6585b6c6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"
Jul 29 15:53:31.356825 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.356804    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/147ac01f-e082-4b0a-8113-789f6585b6c6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt\" (UID: \"147ac01f-e082-4b0a-8113-789f6585b6c6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"
Jul 29 15:53:31.356861 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.356820    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/147ac01f-e082-4b0a-8113-789f6585b6c6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt\" (UID: \"147ac01f-e082-4b0a-8113-789f6585b6c6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"
Jul 29 15:53:31.365817 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.365787    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8s5p\" (UniqueName: \"kubernetes.io/projected/147ac01f-e082-4b0a-8113-789f6585b6c6-kube-api-access-t8s5p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt\" (UID: \"147ac01f-e082-4b0a-8113-789f6585b6c6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"
Jul 29 15:53:31.447466 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.447439    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"
Jul 29 15:53:31.580165 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.580131    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"]
Jul 29 15:53:31.582470 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:53:31.582446    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod147ac01f_e082_4b0a_8113_789f6585b6c6.slice/crio-2564f07ffdce9afdb8c59b9cce946d88b235b564d0c997008f6f1f68c22d86df WatchSource:0}: Error finding container 2564f07ffdce9afdb8c59b9cce946d88b235b564d0c997008f6f1f68c22d86df: Status 404 returned error can't find the container with id 2564f07ffdce9afdb8c59b9cce946d88b235b564d0c997008f6f1f68c22d86df
Jul 29 15:53:31.932050 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.932012    2560 generic.go:356] "Generic (PLEG): container finished" podID="147ac01f-e082-4b0a-8113-789f6585b6c6" containerID="110df6250ff28bdad605589ddbcd27c415bc55b83f184d10c307aaa868f6d792" exitCode=0
Jul 29 15:53:31.932245 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.932093    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt" event={"ID":"147ac01f-e082-4b0a-8113-789f6585b6c6","Type":"ContainerDied","Data":"110df6250ff28bdad605589ddbcd27c415bc55b83f184d10c307aaa868f6d792"}
Jul 29 15:53:31.932245 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:31.932171    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt" event={"ID":"147ac01f-e082-4b0a-8113-789f6585b6c6","Type":"ContainerStarted","Data":"2564f07ffdce9afdb8c59b9cce946d88b235b564d0c997008f6f1f68c22d86df"}
Jul 29 15:53:32.938829 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:32.938742    2560 generic.go:356] "Generic (PLEG): container finished" podID="147ac01f-e082-4b0a-8113-789f6585b6c6" containerID="9c9522626188ee1bf3be3eaa0ca33c4dc86f4a7ee8e1745e46da4c5537f25fda" exitCode=0
Jul 29 15:53:32.939257 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:32.938822    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt" event={"ID":"147ac01f-e082-4b0a-8113-789f6585b6c6","Type":"ContainerDied","Data":"9c9522626188ee1bf3be3eaa0ca33c4dc86f4a7ee8e1745e46da4c5537f25fda"}
Jul 29 15:53:33.945264 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:33.945231    2560 generic.go:356] "Generic (PLEG): container finished" podID="147ac01f-e082-4b0a-8113-789f6585b6c6" containerID="83e22eb6926468d9fefde31df2a19000d006f5ad930dbfb9f9c2e715b2778d48" exitCode=0
Jul 29 15:53:33.945639 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:33.945317    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt" event={"ID":"147ac01f-e082-4b0a-8113-789f6585b6c6","Type":"ContainerDied","Data":"83e22eb6926468d9fefde31df2a19000d006f5ad930dbfb9f9c2e715b2778d48"}
Jul 29 15:53:35.078194 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:35.078169    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"
Jul 29 15:53:35.189547 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:35.189506    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/147ac01f-e082-4b0a-8113-789f6585b6c6-util\") pod \"147ac01f-e082-4b0a-8113-789f6585b6c6\" (UID: \"147ac01f-e082-4b0a-8113-789f6585b6c6\") "
Jul 29 15:53:35.189547 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:35.189546    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8s5p\" (UniqueName: \"kubernetes.io/projected/147ac01f-e082-4b0a-8113-789f6585b6c6-kube-api-access-t8s5p\") pod \"147ac01f-e082-4b0a-8113-789f6585b6c6\" (UID: \"147ac01f-e082-4b0a-8113-789f6585b6c6\") "
Jul 29 15:53:35.189799 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:35.189641    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/147ac01f-e082-4b0a-8113-789f6585b6c6-bundle\") pod \"147ac01f-e082-4b0a-8113-789f6585b6c6\" (UID: \"147ac01f-e082-4b0a-8113-789f6585b6c6\") "
Jul 29 15:53:35.190271 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:35.190237    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/147ac01f-e082-4b0a-8113-789f6585b6c6-bundle" (OuterVolumeSpecName: "bundle") pod "147ac01f-e082-4b0a-8113-789f6585b6c6" (UID: "147ac01f-e082-4b0a-8113-789f6585b6c6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:53:35.191868 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:35.191844    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/147ac01f-e082-4b0a-8113-789f6585b6c6-kube-api-access-t8s5p" (OuterVolumeSpecName: "kube-api-access-t8s5p") pod "147ac01f-e082-4b0a-8113-789f6585b6c6" (UID: "147ac01f-e082-4b0a-8113-789f6585b6c6"). InnerVolumeSpecName "kube-api-access-t8s5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:53:35.195098 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:35.195072    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/147ac01f-e082-4b0a-8113-789f6585b6c6-util" (OuterVolumeSpecName: "util") pod "147ac01f-e082-4b0a-8113-789f6585b6c6" (UID: "147ac01f-e082-4b0a-8113-789f6585b6c6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:53:35.290496 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:35.290405    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/147ac01f-e082-4b0a-8113-789f6585b6c6-bundle\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:53:35.290496 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:35.290430    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/147ac01f-e082-4b0a-8113-789f6585b6c6-util\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:53:35.290496 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:35.290439    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8s5p\" (UniqueName: \"kubernetes.io/projected/147ac01f-e082-4b0a-8113-789f6585b6c6-kube-api-access-t8s5p\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:53:35.956645 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:35.956612    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt" event={"ID":"147ac01f-e082-4b0a-8113-789f6585b6c6","Type":"ContainerDied","Data":"2564f07ffdce9afdb8c59b9cce946d88b235b564d0c997008f6f1f68c22d86df"}
Jul 29 15:53:35.956645 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:35.956642    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt"
Jul 29 15:53:35.956823 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:53:35.956662    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2564f07ffdce9afdb8c59b9cce946d88b235b564d0c997008f6f1f68c22d86df"
Jul 29 15:54:07.892834 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.892757    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"]
Jul 29 15:54:07.893308 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.893103    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="147ac01f-e082-4b0a-8113-789f6585b6c6" containerName="util"
Jul 29 15:54:07.893308 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.893116    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="147ac01f-e082-4b0a-8113-789f6585b6c6" containerName="util"
Jul 29 15:54:07.893308 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.893127    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="147ac01f-e082-4b0a-8113-789f6585b6c6" containerName="extract"
Jul 29 15:54:07.893308 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.893133    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="147ac01f-e082-4b0a-8113-789f6585b6c6" containerName="extract"
Jul 29 15:54:07.893308 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.893144    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="147ac01f-e082-4b0a-8113-789f6585b6c6" containerName="pull"
Jul 29 15:54:07.893308 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.893150    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="147ac01f-e082-4b0a-8113-789f6585b6c6" containerName="pull"
Jul 29 15:54:07.893308 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.893208    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="147ac01f-e082-4b0a-8113-789f6585b6c6" containerName="extract"
Jul 29 15:54:07.896318 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.896302    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"
Jul 29 15:54:07.899437 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.899413    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 15:54:07.899741 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.899723    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 29 15:54:07.899815 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.899745    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 15:54:07.900794 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.900778    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-52j2g\""
Jul 29 15:54:07.900899 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.900801    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 29 15:54:07.908156 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.908134    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"]
Jul 29 15:54:07.950832 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.950800    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5e90aa8-ef73-4060-94e1-6a1d987856c1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cq2xp\" (UID: \"f5e90aa8-ef73-4060-94e1-6a1d987856c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"
Jul 29 15:54:07.951011 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.950842    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dcd7\" (UniqueName: \"kubernetes.io/projected/f5e90aa8-ef73-4060-94e1-6a1d987856c1-kube-api-access-6dcd7\") pod \"kuadrant-console-plugin-66b5774985-cq2xp\" (UID: \"f5e90aa8-ef73-4060-94e1-6a1d987856c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"
Jul 29 15:54:07.951011 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:07.950872    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f5e90aa8-ef73-4060-94e1-6a1d987856c1-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cq2xp\" (UID: \"f5e90aa8-ef73-4060-94e1-6a1d987856c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"
Jul 29 15:54:08.051862 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:08.051834    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f5e90aa8-ef73-4060-94e1-6a1d987856c1-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cq2xp\" (UID: \"f5e90aa8-ef73-4060-94e1-6a1d987856c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"
Jul 29 15:54:08.052046 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:08.051892    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5e90aa8-ef73-4060-94e1-6a1d987856c1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cq2xp\" (UID: \"f5e90aa8-ef73-4060-94e1-6a1d987856c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"
Jul 29 15:54:08.052046 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:08.051937    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dcd7\" (UniqueName: \"kubernetes.io/projected/f5e90aa8-ef73-4060-94e1-6a1d987856c1-kube-api-access-6dcd7\") pod \"kuadrant-console-plugin-66b5774985-cq2xp\" (UID: \"f5e90aa8-ef73-4060-94e1-6a1d987856c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"
Jul 29 15:54:08.052046 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:54:08.052027    2560 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 29 15:54:08.052151 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:54:08.052101    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e90aa8-ef73-4060-94e1-6a1d987856c1-plugin-serving-cert podName:f5e90aa8-ef73-4060-94e1-6a1d987856c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:54:08.55208249 +0000 UTC m=+452.994288737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/f5e90aa8-ef73-4060-94e1-6a1d987856c1-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-cq2xp" (UID: "f5e90aa8-ef73-4060-94e1-6a1d987856c1") : secret "plugin-serving-cert" not found
Jul 29 15:54:08.052513 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:08.052495    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f5e90aa8-ef73-4060-94e1-6a1d987856c1-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cq2xp\" (UID: \"f5e90aa8-ef73-4060-94e1-6a1d987856c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"
Jul 29 15:54:08.061353 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:08.061324    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dcd7\" (UniqueName: \"kubernetes.io/projected/f5e90aa8-ef73-4060-94e1-6a1d987856c1-kube-api-access-6dcd7\") pod \"kuadrant-console-plugin-66b5774985-cq2xp\" (UID: \"f5e90aa8-ef73-4060-94e1-6a1d987856c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"
Jul 29 15:54:08.556129 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:08.556091    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5e90aa8-ef73-4060-94e1-6a1d987856c1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cq2xp\" (UID: \"f5e90aa8-ef73-4060-94e1-6a1d987856c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"
Jul 29 15:54:08.558422 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:08.558398    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5e90aa8-ef73-4060-94e1-6a1d987856c1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cq2xp\" (UID: \"f5e90aa8-ef73-4060-94e1-6a1d987856c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"
Jul 29 15:54:08.805180 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:08.805131    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"
Jul 29 15:54:08.933409 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:08.933376    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp"]
Jul 29 15:54:08.937639 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:54:08.937612    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5e90aa8_ef73_4060_94e1_6a1d987856c1.slice/crio-4c6a252919bc74275d18e475c81db41f84b6c82ff124013aa0b394da5aab8746 WatchSource:0}: Error finding container 4c6a252919bc74275d18e475c81db41f84b6c82ff124013aa0b394da5aab8746: Status 404 returned error can't find the container with id 4c6a252919bc74275d18e475c81db41f84b6c82ff124013aa0b394da5aab8746
Jul 29 15:54:09.105635 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:09.105552    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp" event={"ID":"f5e90aa8-ef73-4060-94e1-6a1d987856c1","Type":"ContainerStarted","Data":"4c6a252919bc74275d18e475c81db41f84b6c82ff124013aa0b394da5aab8746"}
Jul 29 15:54:31.230550 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:54:31.230514    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp" event={"ID":"f5e90aa8-ef73-4060-94e1-6a1d987856c1","Type":"ContainerStarted","Data":"df997e5c4775289d3bb1b000553e2d1f01c8034e0368b55a0fa43ae3282deed3"}
Jul 29 15:55:02.835034 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:02.834981    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cq2xp" podStartSLOduration=34.430308826 podStartE2EDuration="55.834965412s" totalImagesPullingTime="21.404656586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:54:31.262493919 +0000 UTC m=+475.704700178" watchObservedRunningTime="2026-07-29 15:55:02.834965412 +0000 UTC m=+507.277171672"
Jul 29 15:55:02.836586 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:02.836557    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 15:55:02.867768 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:02.867741    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 15:55:02.867934 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:02.867854    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"
Jul 29 15:55:02.870873 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:02.870851    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 15:55:02.872260 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:02.872237    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 15:55:02.914448 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:02.914417    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmpgz\" (UniqueName: \"kubernetes.io/projected/e959de9a-56d3-4856-9c2b-37fac34a8878-kube-api-access-bmpgz\") pod \"limitador-limitador-54d6d988d5-fmhmp\" (UID: \"e959de9a-56d3-4856-9c2b-37fac34a8878\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"
Jul 29 15:55:02.914586 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:02.914468    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e959de9a-56d3-4856-9c2b-37fac34a8878-config-file\") pod \"limitador-limitador-54d6d988d5-fmhmp\" (UID: \"e959de9a-56d3-4856-9c2b-37fac34a8878\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"
Jul 29 15:55:03.015786 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.015745    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmpgz\" (UniqueName: \"kubernetes.io/projected/e959de9a-56d3-4856-9c2b-37fac34a8878-kube-api-access-bmpgz\") pod \"limitador-limitador-54d6d988d5-fmhmp\" (UID: \"e959de9a-56d3-4856-9c2b-37fac34a8878\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"
Jul 29 15:55:03.016016 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.015795    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e959de9a-56d3-4856-9c2b-37fac34a8878-config-file\") pod \"limitador-limitador-54d6d988d5-fmhmp\" (UID: \"e959de9a-56d3-4856-9c2b-37fac34a8878\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"
Jul 29 15:55:03.016435 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.016414    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e959de9a-56d3-4856-9c2b-37fac34a8878-config-file\") pod \"limitador-limitador-54d6d988d5-fmhmp\" (UID: \"e959de9a-56d3-4856-9c2b-37fac34a8878\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"
Jul 29 15:55:03.028408 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.028383    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmpgz\" (UniqueName: \"kubernetes.io/projected/e959de9a-56d3-4856-9c2b-37fac34a8878-kube-api-access-bmpgz\") pod \"limitador-limitador-54d6d988d5-fmhmp\" (UID: \"e959de9a-56d3-4856-9c2b-37fac34a8878\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"
Jul 29 15:55:03.177819 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.177780    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"
Jul 29 15:55:03.311607 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.311580    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 15:55:03.314029 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:55:03.314002    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode959de9a_56d3_4856_9c2b_37fac34a8878.slice/crio-73a538e29c40a868ac96ffd15681017abaf1b62905fc4c4acc1be1fc09ae0e58 WatchSource:0}: Error finding container 73a538e29c40a868ac96ffd15681017abaf1b62905fc4c4acc1be1fc09ae0e58: Status 404 returned error can't find the container with id 73a538e29c40a868ac96ffd15681017abaf1b62905fc4c4acc1be1fc09ae0e58
Jul 29 15:55:03.331977 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.331956    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-rbq64"]
Jul 29 15:55:03.338873 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.337699    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rbq64"
Jul 29 15:55:03.341096 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.341069    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8dff9\""
Jul 29 15:55:03.342500 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.342480    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rbq64"]
Jul 29 15:55:03.365842 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.365809    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-fmhmp" event={"ID":"e959de9a-56d3-4856-9c2b-37fac34a8878","Type":"ContainerStarted","Data":"73a538e29c40a868ac96ffd15681017abaf1b62905fc4c4acc1be1fc09ae0e58"}
Jul 29 15:55:03.419328 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.419302    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpm9q\" (UniqueName: \"kubernetes.io/projected/73face51-a27f-40fb-9fd5-8bdbf92f54f0-kube-api-access-lpm9q\") pod \"authorino-58cd955675-rbq64\" (UID: \"73face51-a27f-40fb-9fd5-8bdbf92f54f0\") " pod="kuadrant-system/authorino-58cd955675-rbq64"
Jul 29 15:55:03.520405 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.520319    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpm9q\" (UniqueName: \"kubernetes.io/projected/73face51-a27f-40fb-9fd5-8bdbf92f54f0-kube-api-access-lpm9q\") pod \"authorino-58cd955675-rbq64\" (UID: \"73face51-a27f-40fb-9fd5-8bdbf92f54f0\") " pod="kuadrant-system/authorino-58cd955675-rbq64"
Jul 29 15:55:03.535043 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.535009    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpm9q\" (UniqueName: \"kubernetes.io/projected/73face51-a27f-40fb-9fd5-8bdbf92f54f0-kube-api-access-lpm9q\") pod \"authorino-58cd955675-rbq64\" (UID: \"73face51-a27f-40fb-9fd5-8bdbf92f54f0\") " pod="kuadrant-system/authorino-58cd955675-rbq64"
Jul 29 15:55:03.540729 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.540703    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-brjg9"]
Jul 29 15:55:03.544249 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.544230    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-brjg9"
Jul 29 15:55:03.552891 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.552865    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-brjg9"]
Jul 29 15:55:03.621205 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.621176    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtlbh\" (UniqueName: \"kubernetes.io/projected/4f45efa8-8284-4b3d-9dbb-deebf34631f4-kube-api-access-mtlbh\") pod \"authorino-75d556bd77-brjg9\" (UID: \"4f45efa8-8284-4b3d-9dbb-deebf34631f4\") " pod="kuadrant-system/authorino-75d556bd77-brjg9"
Jul 29 15:55:03.649160 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.649127    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rbq64"
Jul 29 15:55:03.722370 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.722335    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtlbh\" (UniqueName: \"kubernetes.io/projected/4f45efa8-8284-4b3d-9dbb-deebf34631f4-kube-api-access-mtlbh\") pod \"authorino-75d556bd77-brjg9\" (UID: \"4f45efa8-8284-4b3d-9dbb-deebf34631f4\") " pod="kuadrant-system/authorino-75d556bd77-brjg9"
Jul 29 15:55:03.733117 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.733084    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtlbh\" (UniqueName: \"kubernetes.io/projected/4f45efa8-8284-4b3d-9dbb-deebf34631f4-kube-api-access-mtlbh\") pod \"authorino-75d556bd77-brjg9\" (UID: \"4f45efa8-8284-4b3d-9dbb-deebf34631f4\") " pod="kuadrant-system/authorino-75d556bd77-brjg9"
Jul 29 15:55:03.775945 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.775903    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rbq64"]
Jul 29 15:55:03.778328 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:55:03.778298    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73face51_a27f_40fb_9fd5_8bdbf92f54f0.slice/crio-225fc0e223871631f6dbad6bcd28d99cbdcb088eb38493bc25d385ed6a6b32c0 WatchSource:0}: Error finding container 225fc0e223871631f6dbad6bcd28d99cbdcb088eb38493bc25d385ed6a6b32c0: Status 404 returned error can't find the container with id 225fc0e223871631f6dbad6bcd28d99cbdcb088eb38493bc25d385ed6a6b32c0
Jul 29 15:55:03.853654 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:03.853620    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-brjg9"
Jul 29 15:55:04.020761 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:04.020689    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-brjg9"]
Jul 29 15:55:04.022651 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:55:04.022620    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f45efa8_8284_4b3d_9dbb_deebf34631f4.slice/crio-ee7b78c75b3ac55a686a6411f1c13d9034537b580deba4ab8ca8dcebbb00424f WatchSource:0}: Error finding container ee7b78c75b3ac55a686a6411f1c13d9034537b580deba4ab8ca8dcebbb00424f: Status 404 returned error can't find the container with id ee7b78c75b3ac55a686a6411f1c13d9034537b580deba4ab8ca8dcebbb00424f
Jul 29 15:55:04.375379 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:04.375285    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rbq64" event={"ID":"73face51-a27f-40fb-9fd5-8bdbf92f54f0","Type":"ContainerStarted","Data":"225fc0e223871631f6dbad6bcd28d99cbdcb088eb38493bc25d385ed6a6b32c0"}
Jul 29 15:55:04.376537 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:04.376508    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-brjg9" event={"ID":"4f45efa8-8284-4b3d-9dbb-deebf34631f4","Type":"ContainerStarted","Data":"ee7b78c75b3ac55a686a6411f1c13d9034537b580deba4ab8ca8dcebbb00424f"}
Jul 29 15:55:09.409378 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:09.409336    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-brjg9" event={"ID":"4f45efa8-8284-4b3d-9dbb-deebf34631f4","Type":"ContainerStarted","Data":"eb4d3fac223949f4e0464847838022870c0c594adfc4c529aa14d6dd68a5ba60"}
Jul 29 15:55:09.410775 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:09.410753    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-fmhmp" event={"ID":"e959de9a-56d3-4856-9c2b-37fac34a8878","Type":"ContainerStarted","Data":"ef01e87a6315330fbd4436d5a19954b3cfda1864b1e46a116cd5fadfe29ebfee"}
Jul 29 15:55:09.410905 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:09.410839    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"
Jul 29 15:55:09.411965 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:09.411945    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rbq64" event={"ID":"73face51-a27f-40fb-9fd5-8bdbf92f54f0","Type":"ContainerStarted","Data":"8f6c3323fd20e858b5a95c2682933c195fba96b8be3c3af1daffc037a6c0cc00"}
Jul 29 15:55:09.425754 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:09.425710    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-brjg9" podStartSLOduration=2.018186688 podStartE2EDuration="6.425697724s" totalImagesPullingTime="4.407511036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:55:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:55:09.424946617 +0000 UTC m=+513.867152873" watchObservedRunningTime="2026-07-29 15:55:09.425697724 +0000 UTC m=+513.867903961"
Jul 29 15:55:09.443532 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:09.443488    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-fmhmp" podStartSLOduration=2.338669606 podStartE2EDuration="7.44347351s" totalImagesPullingTime="5.104803904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:55:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:55:09.441740548 +0000 UTC m=+513.883946806" watchObservedRunningTime="2026-07-29 15:55:09.44347351 +0000 UTC m=+513.885679769"
Jul 29 15:55:09.452606 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:09.452571    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rbq64"]
Jul 29 15:55:09.468302 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:09.468258    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-rbq64" podStartSLOduration=1.8267652220000001 podStartE2EDuration="6.468242303s" totalImagesPullingTime="4.641477081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:55:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:55:09.467318562 +0000 UTC m=+513.909524821" watchObservedRunningTime="2026-07-29 15:55:09.468242303 +0000 UTC m=+513.910448561"
Jul 29 15:55:11.421471 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:11.421433    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-rbq64" podUID="73face51-a27f-40fb-9fd5-8bdbf92f54f0" containerName="authorino" containerID="cri-o://8f6c3323fd20e858b5a95c2682933c195fba96b8be3c3af1daffc037a6c0cc00" gracePeriod=30
Jul 29 15:55:11.664986 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:11.664960    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rbq64"
Jul 29 15:55:11.794322 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:11.794221    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpm9q\" (UniqueName: \"kubernetes.io/projected/73face51-a27f-40fb-9fd5-8bdbf92f54f0-kube-api-access-lpm9q\") pod \"73face51-a27f-40fb-9fd5-8bdbf92f54f0\" (UID: \"73face51-a27f-40fb-9fd5-8bdbf92f54f0\") "
Jul 29 15:55:11.796240 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:11.796207    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73face51-a27f-40fb-9fd5-8bdbf92f54f0-kube-api-access-lpm9q" (OuterVolumeSpecName: "kube-api-access-lpm9q") pod "73face51-a27f-40fb-9fd5-8bdbf92f54f0" (UID: "73face51-a27f-40fb-9fd5-8bdbf92f54f0"). InnerVolumeSpecName "kube-api-access-lpm9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:55:11.895087 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:11.895042    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpm9q\" (UniqueName: \"kubernetes.io/projected/73face51-a27f-40fb-9fd5-8bdbf92f54f0-kube-api-access-lpm9q\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:55:12.428168 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:12.428132    2560 generic.go:356] "Generic (PLEG): container finished" podID="73face51-a27f-40fb-9fd5-8bdbf92f54f0" containerID="8f6c3323fd20e858b5a95c2682933c195fba96b8be3c3af1daffc037a6c0cc00" exitCode=0
Jul 29 15:55:12.428560 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:12.428186    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rbq64"
Jul 29 15:55:12.428560 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:12.428215    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rbq64" event={"ID":"73face51-a27f-40fb-9fd5-8bdbf92f54f0","Type":"ContainerDied","Data":"8f6c3323fd20e858b5a95c2682933c195fba96b8be3c3af1daffc037a6c0cc00"}
Jul 29 15:55:12.428560 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:12.428260    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rbq64" event={"ID":"73face51-a27f-40fb-9fd5-8bdbf92f54f0","Type":"ContainerDied","Data":"225fc0e223871631f6dbad6bcd28d99cbdcb088eb38493bc25d385ed6a6b32c0"}
Jul 29 15:55:12.428560 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:12.428277    2560 scope.go:117] "RemoveContainer" containerID="8f6c3323fd20e858b5a95c2682933c195fba96b8be3c3af1daffc037a6c0cc00"
Jul 29 15:55:12.436310 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:12.436289    2560 scope.go:117] "RemoveContainer" containerID="8f6c3323fd20e858b5a95c2682933c195fba96b8be3c3af1daffc037a6c0cc00"
Jul 29 15:55:12.436591 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:55:12.436567    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f6c3323fd20e858b5a95c2682933c195fba96b8be3c3af1daffc037a6c0cc00\": container with ID starting with 8f6c3323fd20e858b5a95c2682933c195fba96b8be3c3af1daffc037a6c0cc00 not found: ID does not exist" containerID="8f6c3323fd20e858b5a95c2682933c195fba96b8be3c3af1daffc037a6c0cc00"
Jul 29 15:55:12.436708 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:12.436604    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f6c3323fd20e858b5a95c2682933c195fba96b8be3c3af1daffc037a6c0cc00"} err="failed to get container status \"8f6c3323fd20e858b5a95c2682933c195fba96b8be3c3af1daffc037a6c0cc00\": rpc error: code = NotFound desc = could not find container \"8f6c3323fd20e858b5a95c2682933c195fba96b8be3c3af1daffc037a6c0cc00\": container with ID starting with 8f6c3323fd20e858b5a95c2682933c195fba96b8be3c3af1daffc037a6c0cc00 not found: ID does not exist"
Jul 29 15:55:12.445489 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:12.445467    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rbq64"]
Jul 29 15:55:12.448901 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:12.448876    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-rbq64"]
Jul 29 15:55:14.043656 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:14.043625    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73face51-a27f-40fb-9fd5-8bdbf92f54f0" path="/var/lib/kubelet/pods/73face51-a27f-40fb-9fd5-8bdbf92f54f0/volumes"
Jul 29 15:55:20.418444 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:20.418413    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"
Jul 29 15:55:38.305508 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.305477    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-s4rt5"]
Jul 29 15:55:38.305891 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.305806    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73face51-a27f-40fb-9fd5-8bdbf92f54f0" containerName="authorino"
Jul 29 15:55:38.305891 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.305816    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="73face51-a27f-40fb-9fd5-8bdbf92f54f0" containerName="authorino"
Jul 29 15:55:38.305891 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.305873    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="73face51-a27f-40fb-9fd5-8bdbf92f54f0" containerName="authorino"
Jul 29 15:55:38.310213 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.310196    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s4rt5"
Jul 29 15:55:38.313074 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.313055    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 29 15:55:38.318849 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.318819    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-s4rt5"]
Jul 29 15:55:38.351763 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.351738    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-s4rt5"]
Jul 29 15:55:38.351981 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:55:38.351964    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-77bvp tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-s4rt5" podUID="083a8ee4-be5b-4046-84f9-cedb6926f306"
Jul 29 15:55:38.386131 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.386104    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-896687f44-r7pnj"]
Jul 29 15:55:38.389361 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.389345    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-896687f44-r7pnj"
Jul 29 15:55:38.398808 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.398785    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-896687f44-r7pnj"]
Jul 29 15:55:38.412027 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.412001    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77bvp\" (UniqueName: \"kubernetes.io/projected/083a8ee4-be5b-4046-84f9-cedb6926f306-kube-api-access-77bvp\") pod \"authorino-76877994-s4rt5\" (UID: \"083a8ee4-be5b-4046-84f9-cedb6926f306\") " pod="kuadrant-system/authorino-76877994-s4rt5"
Jul 29 15:55:38.412136 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.412118    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/083a8ee4-be5b-4046-84f9-cedb6926f306-tls-cert\") pod \"authorino-76877994-s4rt5\" (UID: \"083a8ee4-be5b-4046-84f9-cedb6926f306\") " pod="kuadrant-system/authorino-76877994-s4rt5"
Jul 29 15:55:38.513311 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.513269    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0678def4-28af-4e80-9498-e869f6872899-tls-cert\") pod \"authorino-896687f44-r7pnj\" (UID: \"0678def4-28af-4e80-9498-e869f6872899\") " pod="kuadrant-system/authorino-896687f44-r7pnj"
Jul 29 15:55:38.513472 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.513325    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdlq9\" (UniqueName: \"kubernetes.io/projected/0678def4-28af-4e80-9498-e869f6872899-kube-api-access-mdlq9\") pod \"authorino-896687f44-r7pnj\" (UID: \"0678def4-28af-4e80-9498-e869f6872899\") " pod="kuadrant-system/authorino-896687f44-r7pnj"
Jul 29 15:55:38.513472 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.513347    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/083a8ee4-be5b-4046-84f9-cedb6926f306-tls-cert\") pod \"authorino-76877994-s4rt5\" (UID: \"083a8ee4-be5b-4046-84f9-cedb6926f306\") " pod="kuadrant-system/authorino-76877994-s4rt5"
Jul 29 15:55:38.513472 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.513383    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77bvp\" (UniqueName: \"kubernetes.io/projected/083a8ee4-be5b-4046-84f9-cedb6926f306-kube-api-access-77bvp\") pod \"authorino-76877994-s4rt5\" (UID: \"083a8ee4-be5b-4046-84f9-cedb6926f306\") " pod="kuadrant-system/authorino-76877994-s4rt5"
Jul 29 15:55:38.515723 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.515699    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/083a8ee4-be5b-4046-84f9-cedb6926f306-tls-cert\") pod \"authorino-76877994-s4rt5\" (UID: \"083a8ee4-be5b-4046-84f9-cedb6926f306\") " pod="kuadrant-system/authorino-76877994-s4rt5"
Jul 29 15:55:38.522496 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.522473    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77bvp\" (UniqueName: \"kubernetes.io/projected/083a8ee4-be5b-4046-84f9-cedb6926f306-kube-api-access-77bvp\") pod \"authorino-76877994-s4rt5\" (UID: \"083a8ee4-be5b-4046-84f9-cedb6926f306\") " pod="kuadrant-system/authorino-76877994-s4rt5"
Jul 29 15:55:38.541869 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.541848    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s4rt5"
Jul 29 15:55:38.548729 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.548710    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s4rt5"
Jul 29 15:55:38.614096 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.614012    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdlq9\" (UniqueName: \"kubernetes.io/projected/0678def4-28af-4e80-9498-e869f6872899-kube-api-access-mdlq9\") pod \"authorino-896687f44-r7pnj\" (UID: \"0678def4-28af-4e80-9498-e869f6872899\") " pod="kuadrant-system/authorino-896687f44-r7pnj"
Jul 29 15:55:38.614236 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.614122    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0678def4-28af-4e80-9498-e869f6872899-tls-cert\") pod \"authorino-896687f44-r7pnj\" (UID: \"0678def4-28af-4e80-9498-e869f6872899\") " pod="kuadrant-system/authorino-896687f44-r7pnj"
Jul 29 15:55:38.616594 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.616571    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0678def4-28af-4e80-9498-e869f6872899-tls-cert\") pod \"authorino-896687f44-r7pnj\" (UID: \"0678def4-28af-4e80-9498-e869f6872899\") " pod="kuadrant-system/authorino-896687f44-r7pnj"
Jul 29 15:55:38.622095 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.622075    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdlq9\" (UniqueName: \"kubernetes.io/projected/0678def4-28af-4e80-9498-e869f6872899-kube-api-access-mdlq9\") pod \"authorino-896687f44-r7pnj\" (UID: \"0678def4-28af-4e80-9498-e869f6872899\") " pod="kuadrant-system/authorino-896687f44-r7pnj"
Jul 29 15:55:38.699002 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.698971    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-896687f44-r7pnj"
Jul 29 15:55:38.715183 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.715152    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/083a8ee4-be5b-4046-84f9-cedb6926f306-tls-cert\") pod \"083a8ee4-be5b-4046-84f9-cedb6926f306\" (UID: \"083a8ee4-be5b-4046-84f9-cedb6926f306\") "
Jul 29 15:55:38.715316 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.715247    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77bvp\" (UniqueName: \"kubernetes.io/projected/083a8ee4-be5b-4046-84f9-cedb6926f306-kube-api-access-77bvp\") pod \"083a8ee4-be5b-4046-84f9-cedb6926f306\" (UID: \"083a8ee4-be5b-4046-84f9-cedb6926f306\") "
Jul 29 15:55:38.717256 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.717228    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/083a8ee4-be5b-4046-84f9-cedb6926f306-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "083a8ee4-be5b-4046-84f9-cedb6926f306" (UID: "083a8ee4-be5b-4046-84f9-cedb6926f306"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:55:38.717345 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.717237    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/083a8ee4-be5b-4046-84f9-cedb6926f306-kube-api-access-77bvp" (OuterVolumeSpecName: "kube-api-access-77bvp") pod "083a8ee4-be5b-4046-84f9-cedb6926f306" (UID: "083a8ee4-be5b-4046-84f9-cedb6926f306"). InnerVolumeSpecName "kube-api-access-77bvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:55:38.815783 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.815757    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77bvp\" (UniqueName: \"kubernetes.io/projected/083a8ee4-be5b-4046-84f9-cedb6926f306-kube-api-access-77bvp\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:55:38.815783 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.815782    2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/083a8ee4-be5b-4046-84f9-cedb6926f306-tls-cert\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:55:38.827262 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:38.827199    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-896687f44-r7pnj"]
Jul 29 15:55:38.829312 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:55:38.829285    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0678def4_28af_4e80_9498_e869f6872899.slice/crio-f3c0ecf211eb9ac53de841edb642f0a80400425524e6ff4f8dbf6e7afca25e20 WatchSource:0}: Error finding container f3c0ecf211eb9ac53de841edb642f0a80400425524e6ff4f8dbf6e7afca25e20: Status 404 returned error can't find the container with id f3c0ecf211eb9ac53de841edb642f0a80400425524e6ff4f8dbf6e7afca25e20
Jul 29 15:55:39.549871 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:39.549844    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s4rt5"
Jul 29 15:55:39.550336 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:39.549872    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-896687f44-r7pnj" event={"ID":"0678def4-28af-4e80-9498-e869f6872899","Type":"ContainerStarted","Data":"a4b6fcb205d0fd62289129e6477ada13d665ce08525c70c0f46c0f3b9417a92d"}
Jul 29 15:55:39.550336 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:39.549941    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-896687f44-r7pnj" event={"ID":"0678def4-28af-4e80-9498-e869f6872899","Type":"ContainerStarted","Data":"f3c0ecf211eb9ac53de841edb642f0a80400425524e6ff4f8dbf6e7afca25e20"}
Jul 29 15:55:39.572771 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:39.572729    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-896687f44-r7pnj" podStartSLOduration=0.997483072 podStartE2EDuration="1.57271581s" totalImagesPullingTime="575.232738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:55:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:55:39.570088933 +0000 UTC m=+544.012295193" watchObservedRunningTime="2026-07-29 15:55:39.57271581 +0000 UTC m=+544.014922068"
Jul 29 15:55:39.602908 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:39.602874    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-s4rt5"]
Jul 29 15:55:39.606078 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:39.606054    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-brjg9"]
Jul 29 15:55:39.606229 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:39.606207    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-brjg9" podUID="4f45efa8-8284-4b3d-9dbb-deebf34631f4" containerName="authorino" containerID="cri-o://eb4d3fac223949f4e0464847838022870c0c594adfc4c529aa14d6dd68a5ba60" gracePeriod=30
Jul 29 15:55:39.608500 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:39.608478    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-s4rt5"]
Jul 29 15:55:39.877544 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:39.877517    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-brjg9"
Jul 29 15:55:39.922413 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:39.922387    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtlbh\" (UniqueName: \"kubernetes.io/projected/4f45efa8-8284-4b3d-9dbb-deebf34631f4-kube-api-access-mtlbh\") pod \"4f45efa8-8284-4b3d-9dbb-deebf34631f4\" (UID: \"4f45efa8-8284-4b3d-9dbb-deebf34631f4\") "
Jul 29 15:55:39.924356 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:39.924324    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f45efa8-8284-4b3d-9dbb-deebf34631f4-kube-api-access-mtlbh" (OuterVolumeSpecName: "kube-api-access-mtlbh") pod "4f45efa8-8284-4b3d-9dbb-deebf34631f4" (UID: "4f45efa8-8284-4b3d-9dbb-deebf34631f4"). InnerVolumeSpecName "kube-api-access-mtlbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:55:40.023346 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:40.023284    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtlbh\" (UniqueName: \"kubernetes.io/projected/4f45efa8-8284-4b3d-9dbb-deebf34631f4-kube-api-access-mtlbh\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:55:40.042903 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:40.042873    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="083a8ee4-be5b-4046-84f9-cedb6926f306" path="/var/lib/kubelet/pods/083a8ee4-be5b-4046-84f9-cedb6926f306/volumes"
Jul 29 15:55:40.556056 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:40.556019    2560 generic.go:356] "Generic (PLEG): container finished" podID="4f45efa8-8284-4b3d-9dbb-deebf34631f4" containerID="eb4d3fac223949f4e0464847838022870c0c594adfc4c529aa14d6dd68a5ba60" exitCode=0
Jul 29 15:55:40.556529 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:40.556082    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-brjg9"
Jul 29 15:55:40.556529 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:40.556106    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-brjg9" event={"ID":"4f45efa8-8284-4b3d-9dbb-deebf34631f4","Type":"ContainerDied","Data":"eb4d3fac223949f4e0464847838022870c0c594adfc4c529aa14d6dd68a5ba60"}
Jul 29 15:55:40.556529 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:40.556146    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-brjg9" event={"ID":"4f45efa8-8284-4b3d-9dbb-deebf34631f4","Type":"ContainerDied","Data":"ee7b78c75b3ac55a686a6411f1c13d9034537b580deba4ab8ca8dcebbb00424f"}
Jul 29 15:55:40.556529 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:40.556163    2560 scope.go:117] "RemoveContainer" containerID="eb4d3fac223949f4e0464847838022870c0c594adfc4c529aa14d6dd68a5ba60"
Jul 29 15:55:40.566165 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:40.566147    2560 scope.go:117] "RemoveContainer" containerID="eb4d3fac223949f4e0464847838022870c0c594adfc4c529aa14d6dd68a5ba60"
Jul 29 15:55:40.566451 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:55:40.566426    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb4d3fac223949f4e0464847838022870c0c594adfc4c529aa14d6dd68a5ba60\": container with ID starting with eb4d3fac223949f4e0464847838022870c0c594adfc4c529aa14d6dd68a5ba60 not found: ID does not exist" containerID="eb4d3fac223949f4e0464847838022870c0c594adfc4c529aa14d6dd68a5ba60"
Jul 29 15:55:40.566509 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:40.566461    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb4d3fac223949f4e0464847838022870c0c594adfc4c529aa14d6dd68a5ba60"} err="failed to get container status \"eb4d3fac223949f4e0464847838022870c0c594adfc4c529aa14d6dd68a5ba60\": rpc error: code = NotFound desc = could not find container \"eb4d3fac223949f4e0464847838022870c0c594adfc4c529aa14d6dd68a5ba60\": container with ID starting with eb4d3fac223949f4e0464847838022870c0c594adfc4c529aa14d6dd68a5ba60 not found: ID does not exist"
Jul 29 15:55:40.572467 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:40.572446    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-brjg9"]
Jul 29 15:55:40.576657 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:40.576632    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-brjg9"]
Jul 29 15:55:42.043033 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:55:42.043002    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f45efa8-8284-4b3d-9dbb-deebf34631f4" path="/var/lib/kubelet/pods/4f45efa8-8284-4b3d-9dbb-deebf34631f4/volumes"
Jul 29 15:56:19.876591 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:19.876551    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 15:56:39.484459 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:39.484424    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 15:56:43.558664 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:43.558627    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 15:56:52.349252 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:52.349199    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 15:56:56.943457 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:56.943421    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"]
Jul 29 15:56:56.943842 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:56.943750    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f45efa8-8284-4b3d-9dbb-deebf34631f4" containerName="authorino"
Jul 29 15:56:56.943842 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:56.943760    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f45efa8-8284-4b3d-9dbb-deebf34631f4" containerName="authorino"
Jul 29 15:56:56.943842 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:56.943815    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f45efa8-8284-4b3d-9dbb-deebf34631f4" containerName="authorino"
Jul 29 15:56:56.946799 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:56.946782    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:56.950675 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:56.950650    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 29 15:56:56.950787 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:56.950651    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-j2td7\""
Jul 29 15:56:56.950787 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:56.950664    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 15:56:56.950944 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:56.950929    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 15:56:56.955909 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:56.955888    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"]
Jul 29 15:56:57.055222 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.055185    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npw9z\" (UniqueName: \"kubernetes.io/projected/e8d1a803-de32-4561-b655-0cca8548e992-kube-api-access-npw9z\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.055222 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.055221    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8d1a803-de32-4561-b655-0cca8548e992-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.055410 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.055251    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8d1a803-de32-4561-b655-0cca8548e992-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.055410 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.055333    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8d1a803-de32-4561-b655-0cca8548e992-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.055410 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.055369    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8d1a803-de32-4561-b655-0cca8548e992-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.055410 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.055396    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8d1a803-de32-4561-b655-0cca8548e992-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.156499 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.156461    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8d1a803-de32-4561-b655-0cca8548e992-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.156657 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.156511    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8d1a803-de32-4561-b655-0cca8548e992-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.156657 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.156542    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8d1a803-de32-4561-b655-0cca8548e992-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.156657 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.156565    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8d1a803-de32-4561-b655-0cca8548e992-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.156657 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.156601    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npw9z\" (UniqueName: \"kubernetes.io/projected/e8d1a803-de32-4561-b655-0cca8548e992-kube-api-access-npw9z\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.156657 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.156621    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8d1a803-de32-4561-b655-0cca8548e992-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.156967 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.156941    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8d1a803-de32-4561-b655-0cca8548e992-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.157037 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.156949    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8d1a803-de32-4561-b655-0cca8548e992-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.157037 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.157012    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8d1a803-de32-4561-b655-0cca8548e992-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.158749 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.158723    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8d1a803-de32-4561-b655-0cca8548e992-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.159010 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.158991    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8d1a803-de32-4561-b655-0cca8548e992-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.164963 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.164944    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npw9z\" (UniqueName: \"kubernetes.io/projected/e8d1a803-de32-4561-b655-0cca8548e992-kube-api-access-npw9z\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk\" (UID: \"e8d1a803-de32-4561-b655-0cca8548e992\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.257850 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.257785    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:56:57.388763 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.388741    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"]
Jul 29 15:56:57.391282 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:56:57.391252    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d1a803_de32_4561_b655_0cca8548e992.slice/crio-3600a1877cfe181ea84aa0f6ec61ccf282995cb71aa47d9c1e72bf5dd036e7b5 WatchSource:0}: Error finding container 3600a1877cfe181ea84aa0f6ec61ccf282995cb71aa47d9c1e72bf5dd036e7b5: Status 404 returned error can't find the container with id 3600a1877cfe181ea84aa0f6ec61ccf282995cb71aa47d9c1e72bf5dd036e7b5
Jul 29 15:56:57.392551 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.392528    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 15:56:57.912910 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:57.912714    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk" event={"ID":"e8d1a803-de32-4561-b655-0cca8548e992","Type":"ContainerStarted","Data":"3600a1877cfe181ea84aa0f6ec61ccf282995cb71aa47d9c1e72bf5dd036e7b5"}
Jul 29 15:56:58.753478 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:58.753378    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 15:56:58.919459 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:58.919424    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk" event={"ID":"e8d1a803-de32-4561-b655-0cca8548e992","Type":"ContainerStarted","Data":"e158d6376be40300832fe85eeb6be5cd1b1f577dc90e105c1a8cbffb8377ae26"}
Jul 29 15:56:58.942216 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:56:58.942145    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk" podStartSLOduration=1.925716926 podStartE2EDuration="2.942121789s" totalImagesPullingTime="1.016404863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:56:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:56:58.939290295 +0000 UTC m=+623.381496555" watchObservedRunningTime="2026-07-29 15:56:58.942121789 +0000 UTC m=+623.384328050"
Jul 29 15:57:05.683464 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.683424    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"]
Jul 29 15:57:05.687245 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.687225    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.690253 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.690233    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 29 15:57:05.698559 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.698539    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"]
Jul 29 15:57:05.839512 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.839481    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/467eecae-082a-4f1f-a632-bd943cb45e5d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.839512 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.839526    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dnsd\" (UniqueName: \"kubernetes.io/projected/467eecae-082a-4f1f-a632-bd943cb45e5d-kube-api-access-7dnsd\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.839778 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.839547    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/467eecae-082a-4f1f-a632-bd943cb45e5d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.839778 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.839698    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/467eecae-082a-4f1f-a632-bd943cb45e5d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.839778 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.839744    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/467eecae-082a-4f1f-a632-bd943cb45e5d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.839888 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.839817    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/467eecae-082a-4f1f-a632-bd943cb45e5d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.854963 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.854928    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 15:57:05.940816 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.940742    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/467eecae-082a-4f1f-a632-bd943cb45e5d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.940816 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.940795    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dnsd\" (UniqueName: \"kubernetes.io/projected/467eecae-082a-4f1f-a632-bd943cb45e5d-kube-api-access-7dnsd\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.941032 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.940839    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/467eecae-082a-4f1f-a632-bd943cb45e5d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.941032 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.940901    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/467eecae-082a-4f1f-a632-bd943cb45e5d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.941032 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.940943    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/467eecae-082a-4f1f-a632-bd943cb45e5d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.941032 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.940969    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/467eecae-082a-4f1f-a632-bd943cb45e5d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.941756 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.941396    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/467eecae-082a-4f1f-a632-bd943cb45e5d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.941756 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.941470    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/467eecae-082a-4f1f-a632-bd943cb45e5d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.941756 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.941590    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/467eecae-082a-4f1f-a632-bd943cb45e5d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.943302 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.943267    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/467eecae-082a-4f1f-a632-bd943cb45e5d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.944097 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.944078    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/467eecae-082a-4f1f-a632-bd943cb45e5d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.949838 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.949812    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dnsd\" (UniqueName: \"kubernetes.io/projected/467eecae-082a-4f1f-a632-bd943cb45e5d-kube-api-access-7dnsd\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr\" (UID: \"467eecae-082a-4f1f-a632-bd943cb45e5d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:05.996852 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:05.996826    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:06.150848 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:06.150821    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"]
Jul 29 15:57:06.154352 ip-10-0-143-118 kubenswrapper[2560]: W0729 15:57:06.154321    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod467eecae_082a_4f1f_a632_bd943cb45e5d.slice/crio-7574fe02ad148ea459d8495c092e3c7c3cf7049449a72828f5ec4a93235cb751 WatchSource:0}: Error finding container 7574fe02ad148ea459d8495c092e3c7c3cf7049449a72828f5ec4a93235cb751: Status 404 returned error can't find the container with id 7574fe02ad148ea459d8495c092e3c7c3cf7049449a72828f5ec4a93235cb751
Jul 29 15:57:06.958418 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:06.958380    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr" event={"ID":"467eecae-082a-4f1f-a632-bd943cb45e5d","Type":"ContainerStarted","Data":"c6954e01c0c9937d329ad4647158bb37ffa21f4291180f686e9851327d41f9dd"}
Jul 29 15:57:06.958418 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:06.958420    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr" event={"ID":"467eecae-082a-4f1f-a632-bd943cb45e5d","Type":"ContainerStarted","Data":"7574fe02ad148ea459d8495c092e3c7c3cf7049449a72828f5ec4a93235cb751"}
Jul 29 15:57:06.979197 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:06.979136    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr" podStartSLOduration=1.7744108619999999 podStartE2EDuration="1.979116921s" totalImagesPullingTime="204.706059ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:57:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:57:06.976854736 +0000 UTC m=+631.419060995" watchObservedRunningTime="2026-07-29 15:57:06.979116921 +0000 UTC m=+631.421323179"
Jul 29 15:57:07.258715 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:07.258631    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:57:07.258715 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:07.258698    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:57:07.273554 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:07.273526    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:57:07.974691 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:07.974658    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk"
Jul 29 15:57:15.997106 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:15.997053    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:15.997106 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:15.997119    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:16.010042 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:16.010015    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:17.017763 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:17.017730    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr"
Jul 29 15:57:17.669477 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:57:17.669445    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 15:58:01.381252 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:01.381218    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-896687f44-r7pnj"]
Jul 29 15:58:01.381756 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:01.381441    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-896687f44-r7pnj" podUID="0678def4-28af-4e80-9498-e869f6872899" containerName="authorino" containerID="cri-o://a4b6fcb205d0fd62289129e6477ada13d665ce08525c70c0f46c0f3b9417a92d" gracePeriod=30
Jul 29 15:58:01.631037 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:01.631014    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-896687f44-r7pnj"
Jul 29 15:58:01.719410 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:01.719378    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0678def4-28af-4e80-9498-e869f6872899-tls-cert\") pod \"0678def4-28af-4e80-9498-e869f6872899\" (UID: \"0678def4-28af-4e80-9498-e869f6872899\") "
Jul 29 15:58:01.719566 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:01.719482    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdlq9\" (UniqueName: \"kubernetes.io/projected/0678def4-28af-4e80-9498-e869f6872899-kube-api-access-mdlq9\") pod \"0678def4-28af-4e80-9498-e869f6872899\" (UID: \"0678def4-28af-4e80-9498-e869f6872899\") "
Jul 29 15:58:01.721477 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:01.721442    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0678def4-28af-4e80-9498-e869f6872899-kube-api-access-mdlq9" (OuterVolumeSpecName: "kube-api-access-mdlq9") pod "0678def4-28af-4e80-9498-e869f6872899" (UID: "0678def4-28af-4e80-9498-e869f6872899"). InnerVolumeSpecName "kube-api-access-mdlq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:58:01.730653 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:01.730625    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0678def4-28af-4e80-9498-e869f6872899-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0678def4-28af-4e80-9498-e869f6872899" (UID: "0678def4-28af-4e80-9498-e869f6872899"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:58:01.821169 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:01.821130    2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0678def4-28af-4e80-9498-e869f6872899-tls-cert\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:58:01.821169 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:01.821158    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdlq9\" (UniqueName: \"kubernetes.io/projected/0678def4-28af-4e80-9498-e869f6872899-kube-api-access-mdlq9\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 15:58:02.219756 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:02.219717    2560 generic.go:356] "Generic (PLEG): container finished" podID="0678def4-28af-4e80-9498-e869f6872899" containerID="a4b6fcb205d0fd62289129e6477ada13d665ce08525c70c0f46c0f3b9417a92d" exitCode=0
Jul 29 15:58:02.219756 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:02.219756    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-896687f44-r7pnj"
Jul 29 15:58:02.220049 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:02.219785    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-896687f44-r7pnj" event={"ID":"0678def4-28af-4e80-9498-e869f6872899","Type":"ContainerDied","Data":"a4b6fcb205d0fd62289129e6477ada13d665ce08525c70c0f46c0f3b9417a92d"}
Jul 29 15:58:02.220049 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:02.219822    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-896687f44-r7pnj" event={"ID":"0678def4-28af-4e80-9498-e869f6872899","Type":"ContainerDied","Data":"f3c0ecf211eb9ac53de841edb642f0a80400425524e6ff4f8dbf6e7afca25e20"}
Jul 29 15:58:02.220049 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:02.219843    2560 scope.go:117] "RemoveContainer" containerID="a4b6fcb205d0fd62289129e6477ada13d665ce08525c70c0f46c0f3b9417a92d"
Jul 29 15:58:02.231534 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:02.231508    2560 scope.go:117] "RemoveContainer" containerID="a4b6fcb205d0fd62289129e6477ada13d665ce08525c70c0f46c0f3b9417a92d"
Jul 29 15:58:02.231834 ip-10-0-143-118 kubenswrapper[2560]: E0729 15:58:02.231814    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4b6fcb205d0fd62289129e6477ada13d665ce08525c70c0f46c0f3b9417a92d\": container with ID starting with a4b6fcb205d0fd62289129e6477ada13d665ce08525c70c0f46c0f3b9417a92d not found: ID does not exist" containerID="a4b6fcb205d0fd62289129e6477ada13d665ce08525c70c0f46c0f3b9417a92d"
Jul 29 15:58:02.231897 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:02.231847    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4b6fcb205d0fd62289129e6477ada13d665ce08525c70c0f46c0f3b9417a92d"} err="failed to get container status \"a4b6fcb205d0fd62289129e6477ada13d665ce08525c70c0f46c0f3b9417a92d\": rpc error: code = NotFound desc = could not find container \"a4b6fcb205d0fd62289129e6477ada13d665ce08525c70c0f46c0f3b9417a92d\": container with ID starting with a4b6fcb205d0fd62289129e6477ada13d665ce08525c70c0f46c0f3b9417a92d not found: ID does not exist"
Jul 29 15:58:02.239513 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:02.239487    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-896687f44-r7pnj"]
Jul 29 15:58:02.245443 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:02.245419    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-896687f44-r7pnj"]
Jul 29 15:58:04.042618 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:04.042582    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0678def4-28af-4e80-9498-e869f6872899" path="/var/lib/kubelet/pods/0678def4-28af-4e80-9498-e869f6872899/volumes"
Jul 29 15:58:29.049010 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:29.048926    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 15:58:38.461622 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:38.461575    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 15:58:49.766394 ip-10-0-143-118 kubenswrapper[2560]: I0729 15:58:49.766357    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:00:31.058632 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:00:31.058557    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:00:42.563301 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:00:42.563266    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:01:16.958302 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:01:16.958263    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:01:26.257456 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:01:26.257422    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:01:33.950771 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:01:33.950689    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:01:41.168710 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:01:41.168679    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:01:52.656404 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:01:52.656369    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:02:02.063080 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:02:02.063039    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:02:14.758627 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:02:14.758592    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:02:24.557517 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:02:24.557483    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:02:46.555248 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:02:46.555212    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:02:58.551290 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:02:58.551211    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:03:21.561597 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:03:21.561563    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:03:32.173744 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:03:32.173709    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:03:42.348394 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:03:42.348357    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:03:58.366159 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:03:58.366122    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:04:08.658039 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:04:08.658004    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:04:24.564332 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:04:24.564295    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:04:33.957566 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:04:33.957482    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:04:45.175166 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:04:45.175133    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:04:55.159995 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:04:55.159960    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:06:36.771577 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:06:36.771539    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:06:48.067762 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:06:48.067728    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:07:04.271082 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:07:04.271046    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:07:15.064481 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:07:15.064448    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:07:32.076165 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:07:32.076089    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:07:43.472886 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:07:43.472851    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:07:54.410041 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:07:54.410008    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:08:03.060163 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:08:03.060126    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:08:14.561638 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:08:14.561602    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:08:25.355936 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:08:25.355886    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:08:32.968054 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:08:32.968018    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:08:53.661871 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:08:53.661834    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:09:01.465941 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:01.465832    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:09:14.573415 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.573383    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"]
Jul 29 16:09:14.573782 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.573731    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0678def4-28af-4e80-9498-e869f6872899" containerName="authorino"
Jul 29 16:09:14.573782 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.573742    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0678def4-28af-4e80-9498-e869f6872899" containerName="authorino"
Jul 29 16:09:14.573851 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.573828    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0678def4-28af-4e80-9498-e869f6872899" containerName="authorino"
Jul 29 16:09:14.576396 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.576380    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.579555 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.579531    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-cdc23d9b-kserve-self-signed-certs\""
Jul 29 16:09:14.598158 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.598137    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"]
Jul 29 16:09:14.658945 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.658896    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-tmp-dir\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.658945 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.658943    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgh6m\" (UniqueName: \"kubernetes.io/projected/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-kube-api-access-zgh6m\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.659125 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.658964    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-model-cache\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.659125 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.659037    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-tls-certs\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.659125 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.659068    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-home\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.659125 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.659085    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-dshm\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.760450 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.760403    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-tls-certs\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.760450 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.760452    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-home\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.760675 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.760468    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-dshm\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.760675 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.760536    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-tmp-dir\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.760675 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.760554    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgh6m\" (UniqueName: \"kubernetes.io/projected/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-kube-api-access-zgh6m\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.760675 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.760572    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-model-cache\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.760931 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.760897    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-home\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.761068 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.760959    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-tmp-dir\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.761132 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.761064    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-model-cache\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.762734 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.762716    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-dshm\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.763028 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.763009    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-tls-certs\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.770270 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.770248    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgh6m\" (UniqueName: \"kubernetes.io/projected/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-kube-api-access-zgh6m\") pod \"e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") " pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:14.885346 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:14.885272    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:15.015600 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:15.015571    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"]
Jul 29 16:09:15.016687 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:09:15.016660    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ded1e5d_29ac_45f8_ae8c_fe989ef7dc1d.slice/crio-5d3ac15b2dc921912c25ed147cc4d50dc37874973bf211857e251a082acd6a4f WatchSource:0}: Error finding container 5d3ac15b2dc921912c25ed147cc4d50dc37874973bf211857e251a082acd6a4f: Status 404 returned error can't find the container with id 5d3ac15b2dc921912c25ed147cc4d50dc37874973bf211857e251a082acd6a4f
Jul 29 16:09:15.344900 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:15.344864    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2" event={"ID":"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d","Type":"ContainerStarted","Data":"5055c01d2ede14d834f53855f7b48417ea761a9a70ea36cebbea8857747dd4a7"}
Jul 29 16:09:15.345081 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:15.344935    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2" event={"ID":"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d","Type":"ContainerStarted","Data":"5d3ac15b2dc921912c25ed147cc4d50dc37874973bf211857e251a082acd6a4f"}
Jul 29 16:09:15.367406 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:15.367363    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2" podStartSLOduration=1.367348799 podStartE2EDuration="1.367348799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:09:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:09:15.365416307 +0000 UTC m=+1359.807622582" watchObservedRunningTime="2026-07-29 16:09:15.367348799 +0000 UTC m=+1359.809555059"
Jul 29 16:09:17.573011 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:17.572977    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:09:24.886123 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:24.886028    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:24.886123 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:24.886095    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:24.898559 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:24.898533    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:25.406607 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:25.406577    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:09:28.266665 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.266629    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"]
Jul 29 16:09:28.270071 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.270053    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.273279 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.273259    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-cdc23d9b-kserve-self-signed-certs\""
Jul 29 16:09:28.280889 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.280867    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"]
Jul 29 16:09:28.374327 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.374291    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-model-cache\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.374327 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.374332    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-tls-certs\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.374574 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.374357    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-dshm\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.374574 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.374375    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-home\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.374574 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.374465    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-tmp-dir\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.374574 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.374502    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkn2m\" (UniqueName: \"kubernetes.io/projected/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-kube-api-access-tkn2m\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.475511 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.475482    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-model-cache\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.475695 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.475520    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-tls-certs\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.475695 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.475544    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-dshm\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.475695 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.475566    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-home\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.475695 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.475596    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-tmp-dir\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.475695 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.475611    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkn2m\" (UniqueName: \"kubernetes.io/projected/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-kube-api-access-tkn2m\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.475995 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.475965    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-model-cache\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.476110 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.476070    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-home\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.476110 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.476087    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-tmp-dir\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.477859 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.477836    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-dshm\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.478192 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.478170    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-tls-certs\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.484725 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.484703    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkn2m\" (UniqueName: \"kubernetes.io/projected/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-kube-api-access-tkn2m\") pod \"e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") " pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.579394 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.579307    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:28.713439 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:28.713417    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"]
Jul 29 16:09:28.715661 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:09:28.715635    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81f28263_1da7_4f39_90c0_e6a3a6b02bdb.slice/crio-05d0401b260012c6b14b152cc8e2b529a2e00c6caae180189be680f884637ffe WatchSource:0}: Error finding container 05d0401b260012c6b14b152cc8e2b529a2e00c6caae180189be680f884637ffe: Status 404 returned error can't find the container with id 05d0401b260012c6b14b152cc8e2b529a2e00c6caae180189be680f884637ffe
Jul 29 16:09:29.411229 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:29.411195    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2" event={"ID":"81f28263-1da7-4f39-90c0-e6a3a6b02bdb","Type":"ContainerStarted","Data":"8f158eff2d94a628377b38486cff10e466a365f45ddc7502128c922f1a7a3d0d"}
Jul 29 16:09:29.411597 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:29.411237    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2" event={"ID":"81f28263-1da7-4f39-90c0-e6a3a6b02bdb","Type":"ContainerStarted","Data":"05d0401b260012c6b14b152cc8e2b529a2e00c6caae180189be680f884637ffe"}
Jul 29 16:09:29.433132 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:29.433084    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2" podStartSLOduration=1.43307101 podStartE2EDuration="1.43307101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:09:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:09:29.432129336 +0000 UTC m=+1373.874335595" watchObservedRunningTime="2026-07-29 16:09:29.43307101 +0000 UTC m=+1373.875277269"
Jul 29 16:09:32.966213 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:32.966178    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:09:38.580187 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:38.580151    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:38.580673 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:38.580199    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:38.592571 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:38.592544    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:39.471405 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:39.471380    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:09:59.571742 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:09:59.571707    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"]
Jul 29 16:10:01.431319 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:01.431285    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"]
Jul 29 16:10:02.274295 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:02.274260    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:10:14.590388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:14.590342    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2" podUID="2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d" containerName="main" containerID="cri-o://5055c01d2ede14d834f53855f7b48417ea761a9a70ea36cebbea8857747dd4a7" gracePeriod=45
Jul 29 16:10:14.837782 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:14.837761    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:10:14.983866 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:14.983834    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-home\") pod \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") "
Jul 29 16:10:14.984077 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:14.983873    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgh6m\" (UniqueName: \"kubernetes.io/projected/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-kube-api-access-zgh6m\") pod \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") "
Jul 29 16:10:14.984077 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:14.983899    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-tmp-dir\") pod \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") "
Jul 29 16:10:14.984077 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:14.983934    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-tls-certs\") pod \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") "
Jul 29 16:10:14.984077 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:14.983965    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-model-cache\") pod \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") "
Jul 29 16:10:14.984077 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:14.983994    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-dshm\") pod \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\" (UID: \"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d\") "
Jul 29 16:10:14.984324 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:14.984182    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d" (UID: "2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:10:14.984324 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:14.984199    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-home" (OuterVolumeSpecName: "home") pod "2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d" (UID: "2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:10:14.984324 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:14.984290    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-model-cache" (OuterVolumeSpecName: "model-cache") pod "2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d" (UID: "2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:10:14.986194 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:14.986162    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-dshm" (OuterVolumeSpecName: "dshm") pod "2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d" (UID: "2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:10:14.986194 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:14.986169    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d" (UID: "2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:10:14.986334 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:14.986206    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-kube-api-access-zgh6m" (OuterVolumeSpecName: "kube-api-access-zgh6m") pod "2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d" (UID: "2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d"). InnerVolumeSpecName "kube-api-access-zgh6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:10:15.085392 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.085364    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-dshm\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:10:15.085392 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.085387    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-home\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:10:15.085392 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.085398    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgh6m\" (UniqueName: \"kubernetes.io/projected/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-kube-api-access-zgh6m\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:10:15.085613 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.085406    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-tmp-dir\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:10:15.085613 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.085415    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-tls-certs\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:10:15.085613 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.085426    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d-model-cache\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:10:15.637127 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.637097    2560 generic.go:356] "Generic (PLEG): container finished" podID="2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d" containerID="5055c01d2ede14d834f53855f7b48417ea761a9a70ea36cebbea8857747dd4a7" exitCode=0
Jul 29 16:10:15.637549 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.637145    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"
Jul 29 16:10:15.637549 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.637174    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2" event={"ID":"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d","Type":"ContainerDied","Data":"5055c01d2ede14d834f53855f7b48417ea761a9a70ea36cebbea8857747dd4a7"}
Jul 29 16:10:15.637549 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.637220    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2" event={"ID":"2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d","Type":"ContainerDied","Data":"5d3ac15b2dc921912c25ed147cc4d50dc37874973bf211857e251a082acd6a4f"}
Jul 29 16:10:15.637549 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.637239    2560 scope.go:117] "RemoveContainer" containerID="5055c01d2ede14d834f53855f7b48417ea761a9a70ea36cebbea8857747dd4a7"
Jul 29 16:10:15.646951 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.646908    2560 scope.go:117] "RemoveContainer" containerID="5055c01d2ede14d834f53855f7b48417ea761a9a70ea36cebbea8857747dd4a7"
Jul 29 16:10:15.647200 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:10:15.647181    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5055c01d2ede14d834f53855f7b48417ea761a9a70ea36cebbea8857747dd4a7\": container with ID starting with 5055c01d2ede14d834f53855f7b48417ea761a9a70ea36cebbea8857747dd4a7 not found: ID does not exist" containerID="5055c01d2ede14d834f53855f7b48417ea761a9a70ea36cebbea8857747dd4a7"
Jul 29 16:10:15.647244 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.647208    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5055c01d2ede14d834f53855f7b48417ea761a9a70ea36cebbea8857747dd4a7"} err="failed to get container status \"5055c01d2ede14d834f53855f7b48417ea761a9a70ea36cebbea8857747dd4a7\": rpc error: code = NotFound desc = could not find container \"5055c01d2ede14d834f53855f7b48417ea761a9a70ea36cebbea8857747dd4a7\": container with ID starting with 5055c01d2ede14d834f53855f7b48417ea761a9a70ea36cebbea8857747dd4a7 not found: ID does not exist"
Jul 29 16:10:15.661815 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.661792    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"]
Jul 29 16:10:15.669483 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:15.669464    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-cdc23d9b-kserve-65676d485f-kbbr2"]
Jul 29 16:10:16.042543 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.042509    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d" path="/var/lib/kubelet/pods/2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d/volumes"
Jul 29 16:10:16.449765 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.449728    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2" podUID="81f28263-1da7-4f39-90c0-e6a3a6b02bdb" containerName="main" containerID="cri-o://8f158eff2d94a628377b38486cff10e466a365f45ddc7502128c922f1a7a3d0d" gracePeriod=45
Jul 29 16:10:16.646370 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.646336    2560 generic.go:356] "Generic (PLEG): container finished" podID="81f28263-1da7-4f39-90c0-e6a3a6b02bdb" containerID="8f158eff2d94a628377b38486cff10e466a365f45ddc7502128c922f1a7a3d0d" exitCode=0
Jul 29 16:10:16.646732 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.646377    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2" event={"ID":"81f28263-1da7-4f39-90c0-e6a3a6b02bdb","Type":"ContainerDied","Data":"8f158eff2d94a628377b38486cff10e466a365f45ddc7502128c922f1a7a3d0d"}
Jul 29 16:10:16.699296 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.699271    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:10:16.799831 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.799760    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkn2m\" (UniqueName: \"kubernetes.io/projected/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-kube-api-access-tkn2m\") pod \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") "
Jul 29 16:10:16.799831 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.799804    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-tls-certs\") pod \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") "
Jul 29 16:10:16.799831 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.799822    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-home\") pod \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") "
Jul 29 16:10:16.800130 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.799861    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-tmp-dir\") pod \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") "
Jul 29 16:10:16.800130 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.799896    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-model-cache\") pod \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") "
Jul 29 16:10:16.800130 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.799932    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-dshm\") pod \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\" (UID: \"81f28263-1da7-4f39-90c0-e6a3a6b02bdb\") "
Jul 29 16:10:16.800311 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.800163    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-home" (OuterVolumeSpecName: "home") pod "81f28263-1da7-4f39-90c0-e6a3a6b02bdb" (UID: "81f28263-1da7-4f39-90c0-e6a3a6b02bdb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:10:16.800311 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.800185    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "81f28263-1da7-4f39-90c0-e6a3a6b02bdb" (UID: "81f28263-1da7-4f39-90c0-e6a3a6b02bdb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:10:16.800311 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.800210    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-model-cache" (OuterVolumeSpecName: "model-cache") pod "81f28263-1da7-4f39-90c0-e6a3a6b02bdb" (UID: "81f28263-1da7-4f39-90c0-e6a3a6b02bdb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:10:16.801869 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.801844    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-dshm" (OuterVolumeSpecName: "dshm") pod "81f28263-1da7-4f39-90c0-e6a3a6b02bdb" (UID: "81f28263-1da7-4f39-90c0-e6a3a6b02bdb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:10:16.802000 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.801943    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "81f28263-1da7-4f39-90c0-e6a3a6b02bdb" (UID: "81f28263-1da7-4f39-90c0-e6a3a6b02bdb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:10:16.802047 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.802010    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-kube-api-access-tkn2m" (OuterVolumeSpecName: "kube-api-access-tkn2m") pod "81f28263-1da7-4f39-90c0-e6a3a6b02bdb" (UID: "81f28263-1da7-4f39-90c0-e6a3a6b02bdb"). InnerVolumeSpecName "kube-api-access-tkn2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:10:16.900731 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.900704    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkn2m\" (UniqueName: \"kubernetes.io/projected/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-kube-api-access-tkn2m\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:10:16.900731 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.900727    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-tls-certs\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:10:16.900731 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.900737    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-home\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:10:16.900937 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.900745    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-tmp-dir\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:10:16.900937 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.900753    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-model-cache\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:10:16.900937 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:16.900761    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f28263-1da7-4f39-90c0-e6a3a6b02bdb-dshm\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:10:17.657037 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:17.657007    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"
Jul 29 16:10:17.657428 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:17.657040    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2" event={"ID":"81f28263-1da7-4f39-90c0-e6a3a6b02bdb","Type":"ContainerDied","Data":"05d0401b260012c6b14b152cc8e2b529a2e00c6caae180189be680f884637ffe"}
Jul 29 16:10:17.657428 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:17.657088    2560 scope.go:117] "RemoveContainer" containerID="8f158eff2d94a628377b38486cff10e466a365f45ddc7502128c922f1a7a3d0d"
Jul 29 16:10:17.682975 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:17.682950    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"]
Jul 29 16:10:17.689160 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:17.689136    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-cdc23d9b-kserve-bbc6cbfcf-jngv2"]
Jul 29 16:10:18.043241 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:18.043166    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81f28263-1da7-4f39-90c0-e6a3a6b02bdb" path="/var/lib/kubelet/pods/81f28263-1da7-4f39-90c0-e6a3a6b02bdb/volumes"
Jul 29 16:10:18.260390 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:18.260356    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:10:28.765369 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:28.765280    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:10:41.765639 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:41.765601    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:10:56.670375 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:10:56.670344    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:11:06.561494 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:11:06.561461    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:11:19.871678 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:11:19.871640    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:11:29.160509 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:11:29.160470    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:11:43.059590 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:11:43.059559    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:11:52.270863 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:11:52.270828    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:12:07.672476 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:12:07.672400    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:12:15.667928 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:12:15.667891    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:12:28.362687 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:12:28.362652    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:12:45.571151 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:12:45.571121    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:12:57.868881 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:12:57.864577    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:13:11.666730 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:11.666693    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:13:25.075006 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:25.074972    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:13:34.570100 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:34.570025    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:13:45.072202 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:45.072167    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:13:49.262211 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.262176    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"]
Jul 29 16:13:49.262557 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.262507    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81f28263-1da7-4f39-90c0-e6a3a6b02bdb" containerName="main"
Jul 29 16:13:49.262557 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.262518    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="81f28263-1da7-4f39-90c0-e6a3a6b02bdb" containerName="main"
Jul 29 16:13:49.262557 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.262527    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d" containerName="main"
Jul 29 16:13:49.262557 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.262532    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d" containerName="main"
Jul 29 16:13:49.262687 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.262593    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ded1e5d-29ac-45f8-ae8c-fe989ef7dc1d" containerName="main"
Jul 29 16:13:49.262687 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.262600    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="81f28263-1da7-4f39-90c0-e6a3a6b02bdb" containerName="main"
Jul 29 16:13:49.265391 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.265375    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"
Jul 29 16:13:49.268868 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.268849    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-2f353f9d\""
Jul 29 16:13:49.277807 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.277786    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"]
Jul 29 16:13:49.299981 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.299955    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1da93aa5-bd9e-4eac-a786-1cef9d39cf1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt\" (UID: \"1da93aa5-bd9e-4eac-a786-1cef9d39cf1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"
Jul 29 16:13:49.300102 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.300023    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh8hj\" (UniqueName: \"kubernetes.io/projected/1da93aa5-bd9e-4eac-a786-1cef9d39cf1b-kube-api-access-bh8hj\") pod \"payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt\" (UID: \"1da93aa5-bd9e-4eac-a786-1cef9d39cf1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"
Jul 29 16:13:49.400724 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.400688    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1da93aa5-bd9e-4eac-a786-1cef9d39cf1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt\" (UID: \"1da93aa5-bd9e-4eac-a786-1cef9d39cf1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"
Jul 29 16:13:49.400903 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.400778    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh8hj\" (UniqueName: \"kubernetes.io/projected/1da93aa5-bd9e-4eac-a786-1cef9d39cf1b-kube-api-access-bh8hj\") pod \"payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt\" (UID: \"1da93aa5-bd9e-4eac-a786-1cef9d39cf1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"
Jul 29 16:13:49.401395 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.401373    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1da93aa5-bd9e-4eac-a786-1cef9d39cf1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt\" (UID: \"1da93aa5-bd9e-4eac-a786-1cef9d39cf1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"
Jul 29 16:13:49.411152 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.411130    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh8hj\" (UniqueName: \"kubernetes.io/projected/1da93aa5-bd9e-4eac-a786-1cef9d39cf1b-kube-api-access-bh8hj\") pod \"payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt\" (UID: \"1da93aa5-bd9e-4eac-a786-1cef9d39cf1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"
Jul 29 16:13:49.574990 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.574882    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"
Jul 29 16:13:49.704800 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.704769    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"]
Jul 29 16:13:49.708095 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:13:49.708066    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1da93aa5_bd9e_4eac_a786_1cef9d39cf1b.slice/crio-379b7e469c19d480b9e9d72338e7d052c93339d57884140e24478a9ccf4526db WatchSource:0}: Error finding container 379b7e469c19d480b9e9d72338e7d052c93339d57884140e24478a9ccf4526db: Status 404 returned error can't find the container with id 379b7e469c19d480b9e9d72338e7d052c93339d57884140e24478a9ccf4526db
Jul 29 16:13:49.710202 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:49.710182    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 16:13:50.637799 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:50.637759    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt" event={"ID":"1da93aa5-bd9e-4eac-a786-1cef9d39cf1b","Type":"ContainerStarted","Data":"379b7e469c19d480b9e9d72338e7d052c93339d57884140e24478a9ccf4526db"}
Jul 29 16:13:51.646768 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:51.646728    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt" event={"ID":"1da93aa5-bd9e-4eac-a786-1cef9d39cf1b","Type":"ContainerStarted","Data":"61f53ce944fc03bce99628b29aab830c8f2114a5f5e8fa88f8288ecb3885f2d2"}
Jul 29 16:13:51.647182 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:51.646828    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"
Jul 29 16:13:51.666897 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:51.666816    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt" podStartSLOduration=0.977837964 podStartE2EDuration="2.666802838s" totalImagesPullingTime="1.688964874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:13:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:13:51.665527984 +0000 UTC m=+1636.107734242" watchObservedRunningTime="2026-07-29 16:13:51.666802838 +0000 UTC m=+1636.109009096"
Jul 29 16:13:54.162659 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:13:54.162620    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:14:02.653840 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:02.653812    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"
Jul 29 16:14:29.204862 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.204824    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"]
Jul 29 16:14:29.205414 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.205102    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt" podUID="1da93aa5-bd9e-4eac-a786-1cef9d39cf1b" containerName="payload-pre-processing" containerID="cri-o://61f53ce944fc03bce99628b29aab830c8f2114a5f5e8fa88f8288ecb3885f2d2" gracePeriod=30
Jul 29 16:14:29.452762 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.452737    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"
Jul 29 16:14:29.536887 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.536818    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1da93aa5-bd9e-4eac-a786-1cef9d39cf1b-plugins-config-volume\") pod \"1da93aa5-bd9e-4eac-a786-1cef9d39cf1b\" (UID: \"1da93aa5-bd9e-4eac-a786-1cef9d39cf1b\") "
Jul 29 16:14:29.536887 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.536855    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bh8hj\" (UniqueName: \"kubernetes.io/projected/1da93aa5-bd9e-4eac-a786-1cef9d39cf1b-kube-api-access-bh8hj\") pod \"1da93aa5-bd9e-4eac-a786-1cef9d39cf1b\" (UID: \"1da93aa5-bd9e-4eac-a786-1cef9d39cf1b\") "
Jul 29 16:14:29.537311 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.537266    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1da93aa5-bd9e-4eac-a786-1cef9d39cf1b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1da93aa5-bd9e-4eac-a786-1cef9d39cf1b" (UID: "1da93aa5-bd9e-4eac-a786-1cef9d39cf1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:14:29.540504 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.540476    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1da93aa5-bd9e-4eac-a786-1cef9d39cf1b-kube-api-access-bh8hj" (OuterVolumeSpecName: "kube-api-access-bh8hj") pod "1da93aa5-bd9e-4eac-a786-1cef9d39cf1b" (UID: "1da93aa5-bd9e-4eac-a786-1cef9d39cf1b"). InnerVolumeSpecName "kube-api-access-bh8hj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:14:29.638126 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.638093    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1da93aa5-bd9e-4eac-a786-1cef9d39cf1b-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:14:29.638126 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.638121    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh8hj\" (UniqueName: \"kubernetes.io/projected/1da93aa5-bd9e-4eac-a786-1cef9d39cf1b-kube-api-access-bh8hj\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:14:29.828294 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.828212    2560 generic.go:356] "Generic (PLEG): container finished" podID="1da93aa5-bd9e-4eac-a786-1cef9d39cf1b" containerID="61f53ce944fc03bce99628b29aab830c8f2114a5f5e8fa88f8288ecb3885f2d2" exitCode=0
Jul 29 16:14:29.828294 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.828262    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"
Jul 29 16:14:29.828459 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.828296    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt" event={"ID":"1da93aa5-bd9e-4eac-a786-1cef9d39cf1b","Type":"ContainerDied","Data":"61f53ce944fc03bce99628b29aab830c8f2114a5f5e8fa88f8288ecb3885f2d2"}
Jul 29 16:14:29.828459 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.828353    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt" event={"ID":"1da93aa5-bd9e-4eac-a786-1cef9d39cf1b","Type":"ContainerDied","Data":"379b7e469c19d480b9e9d72338e7d052c93339d57884140e24478a9ccf4526db"}
Jul 29 16:14:29.828459 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.828375    2560 scope.go:117] "RemoveContainer" containerID="61f53ce944fc03bce99628b29aab830c8f2114a5f5e8fa88f8288ecb3885f2d2"
Jul 29 16:14:29.837863 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.837848    2560 scope.go:117] "RemoveContainer" containerID="61f53ce944fc03bce99628b29aab830c8f2114a5f5e8fa88f8288ecb3885f2d2"
Jul 29 16:14:29.838164 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:14:29.838147    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61f53ce944fc03bce99628b29aab830c8f2114a5f5e8fa88f8288ecb3885f2d2\": container with ID starting with 61f53ce944fc03bce99628b29aab830c8f2114a5f5e8fa88f8288ecb3885f2d2 not found: ID does not exist" containerID="61f53ce944fc03bce99628b29aab830c8f2114a5f5e8fa88f8288ecb3885f2d2"
Jul 29 16:14:29.838231 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.838174    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61f53ce944fc03bce99628b29aab830c8f2114a5f5e8fa88f8288ecb3885f2d2"} err="failed to get container status \"61f53ce944fc03bce99628b29aab830c8f2114a5f5e8fa88f8288ecb3885f2d2\": rpc error: code = NotFound desc = could not find container \"61f53ce944fc03bce99628b29aab830c8f2114a5f5e8fa88f8288ecb3885f2d2\": container with ID starting with 61f53ce944fc03bce99628b29aab830c8f2114a5f5e8fa88f8288ecb3885f2d2 not found: ID does not exist"
Jul 29 16:14:29.852227 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.852201    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"]
Jul 29 16:14:29.854338 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:29.854318    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2f353f9d-5d7f8b48bd-s6zkt"]
Jul 29 16:14:30.043665 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:30.043633    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1da93aa5-bd9e-4eac-a786-1cef9d39cf1b" path="/var/lib/kubelet/pods/1da93aa5-bd9e-4eac-a786-1cef9d39cf1b/volumes"
Jul 29 16:14:33.458368 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:33.458332    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:14:34.537600 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.537559    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"]
Jul 29 16:14:34.538165 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.538146    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da93aa5-bd9e-4eac-a786-1cef9d39cf1b" containerName="payload-pre-processing"
Jul 29 16:14:34.538165 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.538166    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da93aa5-bd9e-4eac-a786-1cef9d39cf1b" containerName="payload-pre-processing"
Jul 29 16:14:34.538307 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.538271    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da93aa5-bd9e-4eac-a786-1cef9d39cf1b" containerName="payload-pre-processing"
Jul 29 16:14:34.542697 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.542677    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"
Jul 29 16:14:34.545458 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.545425    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-959c7923\""
Jul 29 16:14:34.545564 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.545544    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-959c7923-dockercfg-cp586\""
Jul 29 16:14:34.554548 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.554529    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"]
Jul 29 16:14:34.594466 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.594436    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"]
Jul 29 16:14:34.597593 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.597574    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"
Jul 29 16:14:34.606804 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.606785    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"]
Jul 29 16:14:34.684190 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.684161    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eee416c-9559-400d-8589-fd963d5b64ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t\" (UID: \"3eee416c-9559-400d-8589-fd963d5b64ae\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"
Jul 29 16:14:34.684339 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.684222    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04a92db6-0cc9-409d-b1b0-0d13bfaf086d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5\" (UID: \"04a92db6-0cc9-409d-b1b0-0d13bfaf086d\") " pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"
Jul 29 16:14:34.684339 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.684270    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn2n8\" (UniqueName: \"kubernetes.io/projected/04a92db6-0cc9-409d-b1b0-0d13bfaf086d-kube-api-access-hn2n8\") pod \"payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5\" (UID: \"04a92db6-0cc9-409d-b1b0-0d13bfaf086d\") " pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"
Jul 29 16:14:34.684339 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.684290    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4l6x\" (UniqueName: \"kubernetes.io/projected/3eee416c-9559-400d-8589-fd963d5b64ae-kube-api-access-k4l6x\") pod \"payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t\" (UID: \"3eee416c-9559-400d-8589-fd963d5b64ae\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"
Jul 29 16:14:34.785355 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.785322    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04a92db6-0cc9-409d-b1b0-0d13bfaf086d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5\" (UID: \"04a92db6-0cc9-409d-b1b0-0d13bfaf086d\") " pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"
Jul 29 16:14:34.785355 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.785358    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn2n8\" (UniqueName: \"kubernetes.io/projected/04a92db6-0cc9-409d-b1b0-0d13bfaf086d-kube-api-access-hn2n8\") pod \"payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5\" (UID: \"04a92db6-0cc9-409d-b1b0-0d13bfaf086d\") " pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"
Jul 29 16:14:34.785554 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.785378    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4l6x\" (UniqueName: \"kubernetes.io/projected/3eee416c-9559-400d-8589-fd963d5b64ae-kube-api-access-k4l6x\") pod \"payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t\" (UID: \"3eee416c-9559-400d-8589-fd963d5b64ae\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"
Jul 29 16:14:34.785554 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.785413    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eee416c-9559-400d-8589-fd963d5b64ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t\" (UID: \"3eee416c-9559-400d-8589-fd963d5b64ae\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"
Jul 29 16:14:34.786089 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.786071    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eee416c-9559-400d-8589-fd963d5b64ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t\" (UID: \"3eee416c-9559-400d-8589-fd963d5b64ae\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"
Jul 29 16:14:34.786144 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.786130    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04a92db6-0cc9-409d-b1b0-0d13bfaf086d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5\" (UID: \"04a92db6-0cc9-409d-b1b0-0d13bfaf086d\") " pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"
Jul 29 16:14:34.796783 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.796719    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4l6x\" (UniqueName: \"kubernetes.io/projected/3eee416c-9559-400d-8589-fd963d5b64ae-kube-api-access-k4l6x\") pod \"payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t\" (UID: \"3eee416c-9559-400d-8589-fd963d5b64ae\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"
Jul 29 16:14:34.797758 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.797742    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn2n8\" (UniqueName: \"kubernetes.io/projected/04a92db6-0cc9-409d-b1b0-0d13bfaf086d-kube-api-access-hn2n8\") pod \"payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5\" (UID: \"04a92db6-0cc9-409d-b1b0-0d13bfaf086d\") " pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"
Jul 29 16:14:34.852449 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.852418    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"
Jul 29 16:14:34.908188 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.908149    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"
Jul 29 16:14:34.995028 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:34.994895    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"]
Jul 29 16:14:34.997732 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:14:34.997698    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a92db6_0cc9_409d_b1b0_0d13bfaf086d.slice/crio-7215a96147e31904f5c712875ab0f46f9c8b5b12a92b914dcd239a690f893624 WatchSource:0}: Error finding container 7215a96147e31904f5c712875ab0f46f9c8b5b12a92b914dcd239a690f893624: Status 404 returned error can't find the container with id 7215a96147e31904f5c712875ab0f46f9c8b5b12a92b914dcd239a690f893624
Jul 29 16:14:35.056980 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:35.056940    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"]
Jul 29 16:14:35.058808 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:14:35.058773    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eee416c_9559_400d_8589_fd963d5b64ae.slice/crio-eb01c42b08bbb7e15853f804dfb011c27aa276fbd335b7a316a95b5c313ff0c8 WatchSource:0}: Error finding container eb01c42b08bbb7e15853f804dfb011c27aa276fbd335b7a316a95b5c313ff0c8: Status 404 returned error can't find the container with id eb01c42b08bbb7e15853f804dfb011c27aa276fbd335b7a316a95b5c313ff0c8
Jul 29 16:14:35.858632 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:35.858595    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5" event={"ID":"04a92db6-0cc9-409d-b1b0-0d13bfaf086d","Type":"ContainerStarted","Data":"34332f3a8676b9408d089d1c336194ba6aa3ddd55f75c55bf1c6fb69e8b05f06"}
Jul 29 16:14:35.859120 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:35.858696    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"
Jul 29 16:14:35.859120 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:35.858723    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5" event={"ID":"04a92db6-0cc9-409d-b1b0-0d13bfaf086d","Type":"ContainerStarted","Data":"7215a96147e31904f5c712875ab0f46f9c8b5b12a92b914dcd239a690f893624"}
Jul 29 16:14:35.860206 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:35.860185    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t" event={"ID":"3eee416c-9559-400d-8589-fd963d5b64ae","Type":"ContainerStarted","Data":"1840bacfca8033dc2f4bac2fba8adbf8a05f0353a0e508dfb5d1221ba99d82b1"}
Jul 29 16:14:35.860305 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:35.860221    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t" event={"ID":"3eee416c-9559-400d-8589-fd963d5b64ae","Type":"ContainerStarted","Data":"eb01c42b08bbb7e15853f804dfb011c27aa276fbd335b7a316a95b5c313ff0c8"}
Jul 29 16:14:35.860305 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:35.860260    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"
Jul 29 16:14:35.877784 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:35.877735    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5" podStartSLOduration=1.8777188580000002 podStartE2EDuration="1.877718858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:14:35.877379255 +0000 UTC m=+1680.319585513" watchObservedRunningTime="2026-07-29 16:14:35.877718858 +0000 UTC m=+1680.319925112"
Jul 29 16:14:35.899599 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:35.899553    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t" podStartSLOduration=1.899535719 podStartE2EDuration="1.899535719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:14:35.898336975 +0000 UTC m=+1680.340543236" watchObservedRunningTime="2026-07-29 16:14:35.899535719 +0000 UTC m=+1680.341741982"
Jul 29 16:14:45.362660 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:45.362626    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:14:46.866580 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:46.866549    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"
Jul 29 16:14:46.867017 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:14:46.866686    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"
Jul 29 16:15:05.484593 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.484508    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"]
Jul 29 16:15:05.485168 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.484744    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5" podUID="04a92db6-0cc9-409d-b1b0-0d13bfaf086d" containerName="payload-processing" containerID="cri-o://34332f3a8676b9408d089d1c336194ba6aa3ddd55f75c55bf1c6fb69e8b05f06" gracePeriod=30
Jul 29 16:15:05.494310 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.494281    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"]
Jul 29 16:15:05.494526 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.494501    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t" podUID="3eee416c-9559-400d-8589-fd963d5b64ae" containerName="payload-pre-processing" containerID="cri-o://1840bacfca8033dc2f4bac2fba8adbf8a05f0353a0e508dfb5d1221ba99d82b1" gracePeriod=30
Jul 29 16:15:05.756989 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.756953    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"
Jul 29 16:15:05.759422 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.759406    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"
Jul 29 16:15:05.830246 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.830218    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04a92db6-0cc9-409d-b1b0-0d13bfaf086d-plugins-config-volume\") pod \"04a92db6-0cc9-409d-b1b0-0d13bfaf086d\" (UID: \"04a92db6-0cc9-409d-b1b0-0d13bfaf086d\") "
Jul 29 16:15:05.830485 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.830465    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4l6x\" (UniqueName: \"kubernetes.io/projected/3eee416c-9559-400d-8589-fd963d5b64ae-kube-api-access-k4l6x\") pod \"3eee416c-9559-400d-8589-fd963d5b64ae\" (UID: \"3eee416c-9559-400d-8589-fd963d5b64ae\") "
Jul 29 16:15:05.830649 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.830616    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eee416c-9559-400d-8589-fd963d5b64ae-plugins-config-volume\") pod \"3eee416c-9559-400d-8589-fd963d5b64ae\" (UID: \"3eee416c-9559-400d-8589-fd963d5b64ae\") "
Jul 29 16:15:05.830757 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.830666    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hn2n8\" (UniqueName: \"kubernetes.io/projected/04a92db6-0cc9-409d-b1b0-0d13bfaf086d-kube-api-access-hn2n8\") pod \"04a92db6-0cc9-409d-b1b0-0d13bfaf086d\" (UID: \"04a92db6-0cc9-409d-b1b0-0d13bfaf086d\") "
Jul 29 16:15:05.831276 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.831075    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eee416c-9559-400d-8589-fd963d5b64ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3eee416c-9559-400d-8589-fd963d5b64ae" (UID: "3eee416c-9559-400d-8589-fd963d5b64ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:15:05.831276 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.831157    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eee416c-9559-400d-8589-fd963d5b64ae-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:15:05.831435 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.831367    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04a92db6-0cc9-409d-b1b0-0d13bfaf086d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "04a92db6-0cc9-409d-b1b0-0d13bfaf086d" (UID: "04a92db6-0cc9-409d-b1b0-0d13bfaf086d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:15:05.833855 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.833820    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eee416c-9559-400d-8589-fd963d5b64ae-kube-api-access-k4l6x" (OuterVolumeSpecName: "kube-api-access-k4l6x") pod "3eee416c-9559-400d-8589-fd963d5b64ae" (UID: "3eee416c-9559-400d-8589-fd963d5b64ae"). InnerVolumeSpecName "kube-api-access-k4l6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:15:05.834151 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.834127    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04a92db6-0cc9-409d-b1b0-0d13bfaf086d-kube-api-access-hn2n8" (OuterVolumeSpecName: "kube-api-access-hn2n8") pod "04a92db6-0cc9-409d-b1b0-0d13bfaf086d" (UID: "04a92db6-0cc9-409d-b1b0-0d13bfaf086d"). InnerVolumeSpecName "kube-api-access-hn2n8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:15:05.931987 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.931956    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4l6x\" (UniqueName: \"kubernetes.io/projected/3eee416c-9559-400d-8589-fd963d5b64ae-kube-api-access-k4l6x\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:15:05.931987 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.931982    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn2n8\" (UniqueName: \"kubernetes.io/projected/04a92db6-0cc9-409d-b1b0-0d13bfaf086d-kube-api-access-hn2n8\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:15:05.931987 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:05.931992    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04a92db6-0cc9-409d-b1b0-0d13bfaf086d-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:15:06.009396 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.009324    2560 generic.go:356] "Generic (PLEG): container finished" podID="3eee416c-9559-400d-8589-fd963d5b64ae" containerID="1840bacfca8033dc2f4bac2fba8adbf8a05f0353a0e508dfb5d1221ba99d82b1" exitCode=0
Jul 29 16:15:06.009396 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.009365    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"
Jul 29 16:15:06.009607 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.009415    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t" event={"ID":"3eee416c-9559-400d-8589-fd963d5b64ae","Type":"ContainerDied","Data":"1840bacfca8033dc2f4bac2fba8adbf8a05f0353a0e508dfb5d1221ba99d82b1"}
Jul 29 16:15:06.009607 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.009501    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t" event={"ID":"3eee416c-9559-400d-8589-fd963d5b64ae","Type":"ContainerDied","Data":"eb01c42b08bbb7e15853f804dfb011c27aa276fbd335b7a316a95b5c313ff0c8"}
Jul 29 16:15:06.009607 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.009536    2560 scope.go:117] "RemoveContainer" containerID="1840bacfca8033dc2f4bac2fba8adbf8a05f0353a0e508dfb5d1221ba99d82b1"
Jul 29 16:15:06.010827 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.010806    2560 generic.go:356] "Generic (PLEG): container finished" podID="04a92db6-0cc9-409d-b1b0-0d13bfaf086d" containerID="34332f3a8676b9408d089d1c336194ba6aa3ddd55f75c55bf1c6fb69e8b05f06" exitCode=0
Jul 29 16:15:06.010934 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.010831    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"
Jul 29 16:15:06.010934 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.010869    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5" event={"ID":"04a92db6-0cc9-409d-b1b0-0d13bfaf086d","Type":"ContainerDied","Data":"34332f3a8676b9408d089d1c336194ba6aa3ddd55f75c55bf1c6fb69e8b05f06"}
Jul 29 16:15:06.010934 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.010906    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5" event={"ID":"04a92db6-0cc9-409d-b1b0-0d13bfaf086d","Type":"ContainerDied","Data":"7215a96147e31904f5c712875ab0f46f9c8b5b12a92b914dcd239a690f893624"}
Jul 29 16:15:06.018280 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.018260    2560 scope.go:117] "RemoveContainer" containerID="1840bacfca8033dc2f4bac2fba8adbf8a05f0353a0e508dfb5d1221ba99d82b1"
Jul 29 16:15:06.018556 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:15:06.018533    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1840bacfca8033dc2f4bac2fba8adbf8a05f0353a0e508dfb5d1221ba99d82b1\": container with ID starting with 1840bacfca8033dc2f4bac2fba8adbf8a05f0353a0e508dfb5d1221ba99d82b1 not found: ID does not exist" containerID="1840bacfca8033dc2f4bac2fba8adbf8a05f0353a0e508dfb5d1221ba99d82b1"
Jul 29 16:15:06.018622 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.018570    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1840bacfca8033dc2f4bac2fba8adbf8a05f0353a0e508dfb5d1221ba99d82b1"} err="failed to get container status \"1840bacfca8033dc2f4bac2fba8adbf8a05f0353a0e508dfb5d1221ba99d82b1\": rpc error: code = NotFound desc = could not find container \"1840bacfca8033dc2f4bac2fba8adbf8a05f0353a0e508dfb5d1221ba99d82b1\": container with ID starting with 1840bacfca8033dc2f4bac2fba8adbf8a05f0353a0e508dfb5d1221ba99d82b1 not found: ID does not exist"
Jul 29 16:15:06.018622 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.018594    2560 scope.go:117] "RemoveContainer" containerID="34332f3a8676b9408d089d1c336194ba6aa3ddd55f75c55bf1c6fb69e8b05f06"
Jul 29 16:15:06.026268 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.026253    2560 scope.go:117] "RemoveContainer" containerID="34332f3a8676b9408d089d1c336194ba6aa3ddd55f75c55bf1c6fb69e8b05f06"
Jul 29 16:15:06.026491 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:15:06.026471    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34332f3a8676b9408d089d1c336194ba6aa3ddd55f75c55bf1c6fb69e8b05f06\": container with ID starting with 34332f3a8676b9408d089d1c336194ba6aa3ddd55f75c55bf1c6fb69e8b05f06 not found: ID does not exist" containerID="34332f3a8676b9408d089d1c336194ba6aa3ddd55f75c55bf1c6fb69e8b05f06"
Jul 29 16:15:06.026544 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.026499    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34332f3a8676b9408d089d1c336194ba6aa3ddd55f75c55bf1c6fb69e8b05f06"} err="failed to get container status \"34332f3a8676b9408d089d1c336194ba6aa3ddd55f75c55bf1c6fb69e8b05f06\": rpc error: code = NotFound desc = could not find container \"34332f3a8676b9408d089d1c336194ba6aa3ddd55f75c55bf1c6fb69e8b05f06\": container with ID starting with 34332f3a8676b9408d089d1c336194ba6aa3ddd55f75c55bf1c6fb69e8b05f06 not found: ID does not exist"
Jul 29 16:15:06.036529 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.036510    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"]
Jul 29 16:15:06.043610 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.043592    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-959c7923-6fbd87ffc7-vnjt5"]
Jul 29 16:15:06.055449 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.055420    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"]
Jul 29 16:15:06.057903 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:06.057880    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-959c7923-9d64bc448-8f82t"]
Jul 29 16:15:08.042588 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:08.042556    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04a92db6-0cc9-409d-b1b0-0d13bfaf086d" path="/var/lib/kubelet/pods/04a92db6-0cc9-409d-b1b0-0d13bfaf086d/volumes"
Jul 29 16:15:08.042982 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:08.042968    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eee416c-9559-400d-8589-fd963d5b64ae" path="/var/lib/kubelet/pods/3eee416c-9559-400d-8589-fd963d5b64ae/volumes"
Jul 29 16:15:09.766895 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.766856    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:15:09.773685 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.773659    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"]
Jul 29 16:15:09.774080 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.774063    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eee416c-9559-400d-8589-fd963d5b64ae" containerName="payload-pre-processing"
Jul 29 16:15:09.774080 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.774082    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eee416c-9559-400d-8589-fd963d5b64ae" containerName="payload-pre-processing"
Jul 29 16:15:09.774226 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.774111    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04a92db6-0cc9-409d-b1b0-0d13bfaf086d" containerName="payload-processing"
Jul 29 16:15:09.774226 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.774120    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="04a92db6-0cc9-409d-b1b0-0d13bfaf086d" containerName="payload-processing"
Jul 29 16:15:09.774226 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.774219    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eee416c-9559-400d-8589-fd963d5b64ae" containerName="payload-pre-processing"
Jul 29 16:15:09.774372 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.774238    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="04a92db6-0cc9-409d-b1b0-0d13bfaf086d" containerName="payload-processing"
Jul 29 16:15:09.777216 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.777198    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"
Jul 29 16:15:09.780140 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.780123    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-d49b3458-dockercfg-sq87h\""
Jul 29 16:15:09.780493 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.780477    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-d49b3458\""
Jul 29 16:15:09.790037 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.790015    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"]
Jul 29 16:15:09.830308 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.830280    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"]
Jul 29 16:15:09.833388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.833373    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"
Jul 29 16:15:09.849319 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.849296    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"]
Jul 29 16:15:09.864887 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.864855    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6cch\" (UniqueName: \"kubernetes.io/projected/fbca3bdd-9a6f-4966-b951-61ca3619fd2b-kube-api-access-r6cch\") pod \"payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt\" (UID: \"fbca3bdd-9a6f-4966-b951-61ca3619fd2b\") " pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"
Jul 29 16:15:09.865092 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.864943    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbca3bdd-9a6f-4966-b951-61ca3619fd2b-plugins-config-volume\") pod \"payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt\" (UID: \"fbca3bdd-9a6f-4966-b951-61ca3619fd2b\") " pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"
Jul 29 16:15:09.865092 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.865040    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdc54909-8a09-4905-875c-abe816dc6286-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g\" (UID: \"cdc54909-8a09-4905-875c-abe816dc6286\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"
Jul 29 16:15:09.865092 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.865070    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pvf2\" (UniqueName: \"kubernetes.io/projected/cdc54909-8a09-4905-875c-abe816dc6286-kube-api-access-5pvf2\") pod \"payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g\" (UID: \"cdc54909-8a09-4905-875c-abe816dc6286\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"
Jul 29 16:15:09.965890 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.965854    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6cch\" (UniqueName: \"kubernetes.io/projected/fbca3bdd-9a6f-4966-b951-61ca3619fd2b-kube-api-access-r6cch\") pod \"payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt\" (UID: \"fbca3bdd-9a6f-4966-b951-61ca3619fd2b\") " pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"
Jul 29 16:15:09.966050 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.965907    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbca3bdd-9a6f-4966-b951-61ca3619fd2b-plugins-config-volume\") pod \"payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt\" (UID: \"fbca3bdd-9a6f-4966-b951-61ca3619fd2b\") " pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"
Jul 29 16:15:09.966050 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.965953    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdc54909-8a09-4905-875c-abe816dc6286-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g\" (UID: \"cdc54909-8a09-4905-875c-abe816dc6286\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"
Jul 29 16:15:09.966050 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.965971    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pvf2\" (UniqueName: \"kubernetes.io/projected/cdc54909-8a09-4905-875c-abe816dc6286-kube-api-access-5pvf2\") pod \"payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g\" (UID: \"cdc54909-8a09-4905-875c-abe816dc6286\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"
Jul 29 16:15:09.966677 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.966651    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdc54909-8a09-4905-875c-abe816dc6286-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g\" (UID: \"cdc54909-8a09-4905-875c-abe816dc6286\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"
Jul 29 16:15:09.966786 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.966704    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbca3bdd-9a6f-4966-b951-61ca3619fd2b-plugins-config-volume\") pod \"payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt\" (UID: \"fbca3bdd-9a6f-4966-b951-61ca3619fd2b\") " pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"
Jul 29 16:15:09.975301 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.975276    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6cch\" (UniqueName: \"kubernetes.io/projected/fbca3bdd-9a6f-4966-b951-61ca3619fd2b-kube-api-access-r6cch\") pod \"payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt\" (UID: \"fbca3bdd-9a6f-4966-b951-61ca3619fd2b\") " pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"
Jul 29 16:15:09.981605 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:09.975992    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pvf2\" (UniqueName: \"kubernetes.io/projected/cdc54909-8a09-4905-875c-abe816dc6286-kube-api-access-5pvf2\") pod \"payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g\" (UID: \"cdc54909-8a09-4905-875c-abe816dc6286\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"
Jul 29 16:15:10.086898 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:10.086821    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"
Jul 29 16:15:10.143506 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:10.143472    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"
Jul 29 16:15:10.284107 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:10.284085    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"]
Jul 29 16:15:10.286715 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:15:10.286688    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdc54909_8a09_4905_875c_abe816dc6286.slice/crio-9706ade055a352abbc4092f74fdceb01c1dfa676374f3ad59a687317e6947f88 WatchSource:0}: Error finding container 9706ade055a352abbc4092f74fdceb01c1dfa676374f3ad59a687317e6947f88: Status 404 returned error can't find the container with id 9706ade055a352abbc4092f74fdceb01c1dfa676374f3ad59a687317e6947f88
Jul 29 16:15:10.444242 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:10.444208    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"]
Jul 29 16:15:10.447337 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:15:10.447309    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbca3bdd_9a6f_4966_b951_61ca3619fd2b.slice/crio-c1b68fdf836d19ec0573a598a7bad2264722ab7a937cb01a82c41e757ce56c57 WatchSource:0}: Error finding container c1b68fdf836d19ec0573a598a7bad2264722ab7a937cb01a82c41e757ce56c57: Status 404 returned error can't find the container with id c1b68fdf836d19ec0573a598a7bad2264722ab7a937cb01a82c41e757ce56c57
Jul 29 16:15:11.036477 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:11.036438    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g" event={"ID":"cdc54909-8a09-4905-875c-abe816dc6286","Type":"ContainerStarted","Data":"893485ca009d00b2015b067e198811f9d0a65c2757e5438c0623b33217a079ad"}
Jul 29 16:15:11.037100 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:11.036500    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g" event={"ID":"cdc54909-8a09-4905-875c-abe816dc6286","Type":"ContainerStarted","Data":"9706ade055a352abbc4092f74fdceb01c1dfa676374f3ad59a687317e6947f88"}
Jul 29 16:15:11.037100 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:11.036529    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"
Jul 29 16:15:11.037969 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:11.037941    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt" event={"ID":"fbca3bdd-9a6f-4966-b951-61ca3619fd2b","Type":"ContainerStarted","Data":"8a2a76ba441fcfbd923c949260e588c5e2221a2c0b661d24c04a8545534dd65c"}
Jul 29 16:15:11.038092 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:11.037999    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"
Jul 29 16:15:11.038092 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:11.038017    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt" event={"ID":"fbca3bdd-9a6f-4966-b951-61ca3619fd2b","Type":"ContainerStarted","Data":"c1b68fdf836d19ec0573a598a7bad2264722ab7a937cb01a82c41e757ce56c57"}
Jul 29 16:15:11.057094 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:11.057045    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g" podStartSLOduration=2.057031831 podStartE2EDuration="2.057031831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:15:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:15:11.056279091 +0000 UTC m=+1715.498485352" watchObservedRunningTime="2026-07-29 16:15:11.057031831 +0000 UTC m=+1715.499238089"
Jul 29 16:15:11.074210 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:11.074164    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt" podStartSLOduration=2.074150263 podStartE2EDuration="2.074150263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:15:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:15:11.073834084 +0000 UTC m=+1715.516040371" watchObservedRunningTime="2026-07-29 16:15:11.074150263 +0000 UTC m=+1715.516356522"
Jul 29 16:15:21.275312 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:21.275279    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:15:22.045089 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:22.045060    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"
Jul 29 16:15:22.045253 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:22.045123    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"
Jul 29 16:15:40.518981 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:40.518951    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"]
Jul 29 16:15:40.519486 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:40.519138    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt" podUID="fbca3bdd-9a6f-4966-b951-61ca3619fd2b" containerName="payload-processing" containerID="cri-o://8a2a76ba441fcfbd923c949260e588c5e2221a2c0b661d24c04a8545534dd65c" gracePeriod=30
Jul 29 16:15:40.529358 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:40.529330    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"]
Jul 29 16:15:40.529535 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:40.529515    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g" podUID="cdc54909-8a09-4905-875c-abe816dc6286" containerName="payload-pre-processing" containerID="cri-o://893485ca009d00b2015b067e198811f9d0a65c2757e5438c0623b33217a079ad" gracePeriod=30
Jul 29 16:15:40.795807 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:40.795769    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"
Jul 29 16:15:40.798655 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:40.798636    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"
Jul 29 16:15:40.929498 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:40.929464    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pvf2\" (UniqueName: \"kubernetes.io/projected/cdc54909-8a09-4905-875c-abe816dc6286-kube-api-access-5pvf2\") pod \"cdc54909-8a09-4905-875c-abe816dc6286\" (UID: \"cdc54909-8a09-4905-875c-abe816dc6286\") "
Jul 29 16:15:40.929671 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:40.929646    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbca3bdd-9a6f-4966-b951-61ca3619fd2b-plugins-config-volume\") pod \"fbca3bdd-9a6f-4966-b951-61ca3619fd2b\" (UID: \"fbca3bdd-9a6f-4966-b951-61ca3619fd2b\") "
Jul 29 16:15:40.929726 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:40.929671    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6cch\" (UniqueName: \"kubernetes.io/projected/fbca3bdd-9a6f-4966-b951-61ca3619fd2b-kube-api-access-r6cch\") pod \"fbca3bdd-9a6f-4966-b951-61ca3619fd2b\" (UID: \"fbca3bdd-9a6f-4966-b951-61ca3619fd2b\") "
Jul 29 16:15:40.929726 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:40.929705    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdc54909-8a09-4905-875c-abe816dc6286-plugins-config-volume\") pod \"cdc54909-8a09-4905-875c-abe816dc6286\" (UID: \"cdc54909-8a09-4905-875c-abe816dc6286\") "
Jul 29 16:15:40.930074 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:40.930042    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbca3bdd-9a6f-4966-b951-61ca3619fd2b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fbca3bdd-9a6f-4966-b951-61ca3619fd2b" (UID: "fbca3bdd-9a6f-4966-b951-61ca3619fd2b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:15:40.930219 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:40.930103    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdc54909-8a09-4905-875c-abe816dc6286-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cdc54909-8a09-4905-875c-abe816dc6286" (UID: "cdc54909-8a09-4905-875c-abe816dc6286"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:15:40.931568 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:40.931544    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdc54909-8a09-4905-875c-abe816dc6286-kube-api-access-5pvf2" (OuterVolumeSpecName: "kube-api-access-5pvf2") pod "cdc54909-8a09-4905-875c-abe816dc6286" (UID: "cdc54909-8a09-4905-875c-abe816dc6286"). InnerVolumeSpecName "kube-api-access-5pvf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:15:40.931741 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:40.931724    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbca3bdd-9a6f-4966-b951-61ca3619fd2b-kube-api-access-r6cch" (OuterVolumeSpecName: "kube-api-access-r6cch") pod "fbca3bdd-9a6f-4966-b951-61ca3619fd2b" (UID: "fbca3bdd-9a6f-4966-b951-61ca3619fd2b"). InnerVolumeSpecName "kube-api-access-r6cch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:15:41.030856 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.030783    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbca3bdd-9a6f-4966-b951-61ca3619fd2b-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:15:41.030856 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.030809    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6cch\" (UniqueName: \"kubernetes.io/projected/fbca3bdd-9a6f-4966-b951-61ca3619fd2b-kube-api-access-r6cch\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:15:41.030856 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.030818    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdc54909-8a09-4905-875c-abe816dc6286-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:15:41.030856 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.030827    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pvf2\" (UniqueName: \"kubernetes.io/projected/cdc54909-8a09-4905-875c-abe816dc6286-kube-api-access-5pvf2\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:15:41.185465 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.185430    2560 generic.go:356] "Generic (PLEG): container finished" podID="cdc54909-8a09-4905-875c-abe816dc6286" containerID="893485ca009d00b2015b067e198811f9d0a65c2757e5438c0623b33217a079ad" exitCode=0
Jul 29 16:15:41.185465 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.185469    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"
Jul 29 16:15:41.185720 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.185517    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g" event={"ID":"cdc54909-8a09-4905-875c-abe816dc6286","Type":"ContainerDied","Data":"893485ca009d00b2015b067e198811f9d0a65c2757e5438c0623b33217a079ad"}
Jul 29 16:15:41.185720 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.185567    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g" event={"ID":"cdc54909-8a09-4905-875c-abe816dc6286","Type":"ContainerDied","Data":"9706ade055a352abbc4092f74fdceb01c1dfa676374f3ad59a687317e6947f88"}
Jul 29 16:15:41.185720 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.185587    2560 scope.go:117] "RemoveContainer" containerID="893485ca009d00b2015b067e198811f9d0a65c2757e5438c0623b33217a079ad"
Jul 29 16:15:41.186775 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.186753    2560 generic.go:356] "Generic (PLEG): container finished" podID="fbca3bdd-9a6f-4966-b951-61ca3619fd2b" containerID="8a2a76ba441fcfbd923c949260e588c5e2221a2c0b661d24c04a8545534dd65c" exitCode=0
Jul 29 16:15:41.186852 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.186783    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt" event={"ID":"fbca3bdd-9a6f-4966-b951-61ca3619fd2b","Type":"ContainerDied","Data":"8a2a76ba441fcfbd923c949260e588c5e2221a2c0b661d24c04a8545534dd65c"}
Jul 29 16:15:41.186852 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.186805    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"
Jul 29 16:15:41.186852 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.186830    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt" event={"ID":"fbca3bdd-9a6f-4966-b951-61ca3619fd2b","Type":"ContainerDied","Data":"c1b68fdf836d19ec0573a598a7bad2264722ab7a937cb01a82c41e757ce56c57"}
Jul 29 16:15:41.194797 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.194781    2560 scope.go:117] "RemoveContainer" containerID="893485ca009d00b2015b067e198811f9d0a65c2757e5438c0623b33217a079ad"
Jul 29 16:15:41.195080 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:15:41.195058    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"893485ca009d00b2015b067e198811f9d0a65c2757e5438c0623b33217a079ad\": container with ID starting with 893485ca009d00b2015b067e198811f9d0a65c2757e5438c0623b33217a079ad not found: ID does not exist" containerID="893485ca009d00b2015b067e198811f9d0a65c2757e5438c0623b33217a079ad"
Jul 29 16:15:41.195150 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.195087    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"893485ca009d00b2015b067e198811f9d0a65c2757e5438c0623b33217a079ad"} err="failed to get container status \"893485ca009d00b2015b067e198811f9d0a65c2757e5438c0623b33217a079ad\": rpc error: code = NotFound desc = could not find container \"893485ca009d00b2015b067e198811f9d0a65c2757e5438c0623b33217a079ad\": container with ID starting with 893485ca009d00b2015b067e198811f9d0a65c2757e5438c0623b33217a079ad not found: ID does not exist"
Jul 29 16:15:41.195150 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.195105    2560 scope.go:117] "RemoveContainer" containerID="8a2a76ba441fcfbd923c949260e588c5e2221a2c0b661d24c04a8545534dd65c"
Jul 29 16:15:41.202871 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.202858    2560 scope.go:117] "RemoveContainer" containerID="8a2a76ba441fcfbd923c949260e588c5e2221a2c0b661d24c04a8545534dd65c"
Jul 29 16:15:41.203115 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:15:41.203099    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a2a76ba441fcfbd923c949260e588c5e2221a2c0b661d24c04a8545534dd65c\": container with ID starting with 8a2a76ba441fcfbd923c949260e588c5e2221a2c0b661d24c04a8545534dd65c not found: ID does not exist" containerID="8a2a76ba441fcfbd923c949260e588c5e2221a2c0b661d24c04a8545534dd65c"
Jul 29 16:15:41.203162 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.203122    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a2a76ba441fcfbd923c949260e588c5e2221a2c0b661d24c04a8545534dd65c"} err="failed to get container status \"8a2a76ba441fcfbd923c949260e588c5e2221a2c0b661d24c04a8545534dd65c\": rpc error: code = NotFound desc = could not find container \"8a2a76ba441fcfbd923c949260e588c5e2221a2c0b661d24c04a8545534dd65c\": container with ID starting with 8a2a76ba441fcfbd923c949260e588c5e2221a2c0b661d24c04a8545534dd65c not found: ID does not exist"
Jul 29 16:15:41.219526 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.219504    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"]
Jul 29 16:15:41.225399 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.225380    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d49b3458-d7c9688db-47lqt"]
Jul 29 16:15:41.239772 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.239751    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"]
Jul 29 16:15:41.244148 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:41.244129    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d49b3458-69586c6bcb-hql6g"]
Jul 29 16:15:42.043079 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:42.043046    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdc54909-8a09-4905-875c-abe816dc6286" path="/var/lib/kubelet/pods/cdc54909-8a09-4905-875c-abe816dc6286/volumes"
Jul 29 16:15:42.043583 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:42.043524    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbca3bdd-9a6f-4966-b951-61ca3619fd2b" path="/var/lib/kubelet/pods/fbca3bdd-9a6f-4966-b951-61ca3619fd2b/volumes"
Jul 29 16:15:44.578380 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:44.578344    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:15:45.857605 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:45.857569    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"]
Jul 29 16:15:45.857998 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:45.857938    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbca3bdd-9a6f-4966-b951-61ca3619fd2b" containerName="payload-processing"
Jul 29 16:15:45.857998 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:45.857955    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbca3bdd-9a6f-4966-b951-61ca3619fd2b" containerName="payload-processing"
Jul 29 16:15:45.857998 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:45.857972    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdc54909-8a09-4905-875c-abe816dc6286" containerName="payload-pre-processing"
Jul 29 16:15:45.857998 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:45.857978    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdc54909-8a09-4905-875c-abe816dc6286" containerName="payload-pre-processing"
Jul 29 16:15:45.858139 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:45.858051    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdc54909-8a09-4905-875c-abe816dc6286" containerName="payload-pre-processing"
Jul 29 16:15:45.858139 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:45.858061    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbca3bdd-9a6f-4966-b951-61ca3619fd2b" containerName="payload-processing"
Jul 29 16:15:45.862452 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:45.862435    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"
Jul 29 16:15:45.865393 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:45.865375    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-facc8562\""
Jul 29 16:15:45.871125 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:45.871102    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"]
Jul 29 16:15:45.972684 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:45.972653    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8b4t\" (UniqueName: \"kubernetes.io/projected/f3413daf-1bed-4eb7-96e3-8590c49139d1-kube-api-access-g8b4t\") pod \"payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn\" (UID: \"f3413daf-1bed-4eb7-96e3-8590c49139d1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"
Jul 29 16:15:45.972684 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:45.972686    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3413daf-1bed-4eb7-96e3-8590c49139d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn\" (UID: \"f3413daf-1bed-4eb7-96e3-8590c49139d1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"
Jul 29 16:15:46.017143 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.017118    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"]
Jul 29 16:15:46.020525 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.020510    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"
Jul 29 16:15:46.023329 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.023303    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-facc8562-dockercfg-nvwv7\""
Jul 29 16:15:46.031579 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.031554    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"]
Jul 29 16:15:46.074126 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.074100    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8b4t\" (UniqueName: \"kubernetes.io/projected/f3413daf-1bed-4eb7-96e3-8590c49139d1-kube-api-access-g8b4t\") pod \"payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn\" (UID: \"f3413daf-1bed-4eb7-96e3-8590c49139d1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"
Jul 29 16:15:46.074280 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.074136    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3413daf-1bed-4eb7-96e3-8590c49139d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn\" (UID: \"f3413daf-1bed-4eb7-96e3-8590c49139d1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"
Jul 29 16:15:46.074969 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.074943    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3413daf-1bed-4eb7-96e3-8590c49139d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn\" (UID: \"f3413daf-1bed-4eb7-96e3-8590c49139d1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"
Jul 29 16:15:46.083197 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.083178    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8b4t\" (UniqueName: \"kubernetes.io/projected/f3413daf-1bed-4eb7-96e3-8590c49139d1-kube-api-access-g8b4t\") pod \"payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn\" (UID: \"f3413daf-1bed-4eb7-96e3-8590c49139d1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"
Jul 29 16:15:46.173600 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.173564    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"
Jul 29 16:15:46.174889 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.174862    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg7cl\" (UniqueName: \"kubernetes.io/projected/cf11ccb0-3442-4215-84ad-3084aa8326a1-kube-api-access-zg7cl\") pod \"payload-processing-e2e-derive-facc8562-65478d848b-85mf8\" (UID: \"cf11ccb0-3442-4215-84ad-3084aa8326a1\") " pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"
Jul 29 16:15:46.175030 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.174953    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf11ccb0-3442-4215-84ad-3084aa8326a1-plugins-config-volume\") pod \"payload-processing-e2e-derive-facc8562-65478d848b-85mf8\" (UID: \"cf11ccb0-3442-4215-84ad-3084aa8326a1\") " pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"
Jul 29 16:15:46.275455 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.275422    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg7cl\" (UniqueName: \"kubernetes.io/projected/cf11ccb0-3442-4215-84ad-3084aa8326a1-kube-api-access-zg7cl\") pod \"payload-processing-e2e-derive-facc8562-65478d848b-85mf8\" (UID: \"cf11ccb0-3442-4215-84ad-3084aa8326a1\") " pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"
Jul 29 16:15:46.275632 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.275470    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf11ccb0-3442-4215-84ad-3084aa8326a1-plugins-config-volume\") pod \"payload-processing-e2e-derive-facc8562-65478d848b-85mf8\" (UID: \"cf11ccb0-3442-4215-84ad-3084aa8326a1\") " pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"
Jul 29 16:15:46.276213 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.276188    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf11ccb0-3442-4215-84ad-3084aa8326a1-plugins-config-volume\") pod \"payload-processing-e2e-derive-facc8562-65478d848b-85mf8\" (UID: \"cf11ccb0-3442-4215-84ad-3084aa8326a1\") " pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"
Jul 29 16:15:46.284747 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.284724    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg7cl\" (UniqueName: \"kubernetes.io/projected/cf11ccb0-3442-4215-84ad-3084aa8326a1-kube-api-access-zg7cl\") pod \"payload-processing-e2e-derive-facc8562-65478d848b-85mf8\" (UID: \"cf11ccb0-3442-4215-84ad-3084aa8326a1\") " pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"
Jul 29 16:15:46.309709 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.309688    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"]
Jul 29 16:15:46.311579 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:15:46.311553    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3413daf_1bed_4eb7_96e3_8590c49139d1.slice/crio-5273e69167ca3fddcf96fda2c1a0a40a20d8b66fb814b6a009ec6d6d23929ed6 WatchSource:0}: Error finding container 5273e69167ca3fddcf96fda2c1a0a40a20d8b66fb814b6a009ec6d6d23929ed6: Status 404 returned error can't find the container with id 5273e69167ca3fddcf96fda2c1a0a40a20d8b66fb814b6a009ec6d6d23929ed6
Jul 29 16:15:46.330745 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.330726    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"
Jul 29 16:15:46.672449 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:46.672416    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"]
Jul 29 16:15:46.674059 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:15:46.674022    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf11ccb0_3442_4215_84ad_3084aa8326a1.slice/crio-eccf907d18f0393343fbcf78d852f9c181d098a14b98a61869c54edbb12c9ae3 WatchSource:0}: Error finding container eccf907d18f0393343fbcf78d852f9c181d098a14b98a61869c54edbb12c9ae3: Status 404 returned error can't find the container with id eccf907d18f0393343fbcf78d852f9c181d098a14b98a61869c54edbb12c9ae3
Jul 29 16:15:47.223801 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:47.223764    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8" event={"ID":"cf11ccb0-3442-4215-84ad-3084aa8326a1","Type":"ContainerStarted","Data":"36118d94d0e43b8a382c0fc93811ad8743703ecb9c61e6fc5e976d513c007236"}
Jul 29 16:15:47.224245 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:47.223826    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8" event={"ID":"cf11ccb0-3442-4215-84ad-3084aa8326a1","Type":"ContainerStarted","Data":"eccf907d18f0393343fbcf78d852f9c181d098a14b98a61869c54edbb12c9ae3"}
Jul 29 16:15:47.224245 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:47.223874    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"
Jul 29 16:15:47.225383 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:47.225363    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn" event={"ID":"f3413daf-1bed-4eb7-96e3-8590c49139d1","Type":"ContainerStarted","Data":"1fa0345c960e89c8b13e53b669ce5d52896c7122e5979e38e6cfd67e1ca4ec28"}
Jul 29 16:15:47.225480 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:47.225395    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn" event={"ID":"f3413daf-1bed-4eb7-96e3-8590c49139d1","Type":"ContainerStarted","Data":"5273e69167ca3fddcf96fda2c1a0a40a20d8b66fb814b6a009ec6d6d23929ed6"}
Jul 29 16:15:47.225480 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:47.225428    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"
Jul 29 16:15:47.306365 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:47.306309    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8" podStartSLOduration=2.306289189 podStartE2EDuration="2.306289189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:15:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:15:47.258953944 +0000 UTC m=+1751.701160203" watchObservedRunningTime="2026-07-29 16:15:47.306289189 +0000 UTC m=+1751.748495451"
Jul 29 16:15:47.307427 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:47.307396    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn" podStartSLOduration=2.307387388 podStartE2EDuration="2.307387388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:15:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:15:47.300622566 +0000 UTC m=+1751.742828826" watchObservedRunningTime="2026-07-29 16:15:47.307387388 +0000 UTC m=+1751.749593651"
Jul 29 16:15:56.254801 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:56.254771    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:15:58.232523 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:58.232490    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"
Jul 29 16:15:58.232992 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:15:58.232647    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"
Jul 29 16:16:16.025322 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.025279    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"]
Jul 29 16:16:16.025845 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.025534    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8" podUID="cf11ccb0-3442-4215-84ad-3084aa8326a1" containerName="payload-processing" containerID="cri-o://36118d94d0e43b8a382c0fc93811ad8743703ecb9c61e6fc5e976d513c007236" gracePeriod=30
Jul 29 16:16:16.045656 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.045637    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"]
Jul 29 16:16:16.045843 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.045822    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn" podUID="f3413daf-1bed-4eb7-96e3-8590c49139d1" containerName="payload-pre-processing" containerID="cri-o://1fa0345c960e89c8b13e53b669ce5d52896c7122e5979e38e6cfd67e1ca4ec28" gracePeriod=30
Jul 29 16:16:16.307204 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.307178    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"
Jul 29 16:16:16.344079 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.344051    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"
Jul 29 16:16:16.366864 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.366832    2560 generic.go:356] "Generic (PLEG): container finished" podID="cf11ccb0-3442-4215-84ad-3084aa8326a1" containerID="36118d94d0e43b8a382c0fc93811ad8743703ecb9c61e6fc5e976d513c007236" exitCode=0
Jul 29 16:16:16.367024 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.366910    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"
Jul 29 16:16:16.367024 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.366939    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8" event={"ID":"cf11ccb0-3442-4215-84ad-3084aa8326a1","Type":"ContainerDied","Data":"36118d94d0e43b8a382c0fc93811ad8743703ecb9c61e6fc5e976d513c007236"}
Jul 29 16:16:16.367024 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.367023    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8" event={"ID":"cf11ccb0-3442-4215-84ad-3084aa8326a1","Type":"ContainerDied","Data":"eccf907d18f0393343fbcf78d852f9c181d098a14b98a61869c54edbb12c9ae3"}
Jul 29 16:16:16.367185 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.367057    2560 scope.go:117] "RemoveContainer" containerID="36118d94d0e43b8a382c0fc93811ad8743703ecb9c61e6fc5e976d513c007236"
Jul 29 16:16:16.368251 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.368224    2560 generic.go:356] "Generic (PLEG): container finished" podID="f3413daf-1bed-4eb7-96e3-8590c49139d1" containerID="1fa0345c960e89c8b13e53b669ce5d52896c7122e5979e38e6cfd67e1ca4ec28" exitCode=0
Jul 29 16:16:16.368369 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.368318    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"
Jul 29 16:16:16.368369 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.368331    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn" event={"ID":"f3413daf-1bed-4eb7-96e3-8590c49139d1","Type":"ContainerDied","Data":"1fa0345c960e89c8b13e53b669ce5d52896c7122e5979e38e6cfd67e1ca4ec28"}
Jul 29 16:16:16.368449 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.368376    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn" event={"ID":"f3413daf-1bed-4eb7-96e3-8590c49139d1","Type":"ContainerDied","Data":"5273e69167ca3fddcf96fda2c1a0a40a20d8b66fb814b6a009ec6d6d23929ed6"}
Jul 29 16:16:16.380822 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.380807    2560 scope.go:117] "RemoveContainer" containerID="36118d94d0e43b8a382c0fc93811ad8743703ecb9c61e6fc5e976d513c007236"
Jul 29 16:16:16.381107 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:16:16.381085    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36118d94d0e43b8a382c0fc93811ad8743703ecb9c61e6fc5e976d513c007236\": container with ID starting with 36118d94d0e43b8a382c0fc93811ad8743703ecb9c61e6fc5e976d513c007236 not found: ID does not exist" containerID="36118d94d0e43b8a382c0fc93811ad8743703ecb9c61e6fc5e976d513c007236"
Jul 29 16:16:16.381178 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.381119    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36118d94d0e43b8a382c0fc93811ad8743703ecb9c61e6fc5e976d513c007236"} err="failed to get container status \"36118d94d0e43b8a382c0fc93811ad8743703ecb9c61e6fc5e976d513c007236\": rpc error: code = NotFound desc = could not find container \"36118d94d0e43b8a382c0fc93811ad8743703ecb9c61e6fc5e976d513c007236\": container with ID starting with 36118d94d0e43b8a382c0fc93811ad8743703ecb9c61e6fc5e976d513c007236 not found: ID does not exist"
Jul 29 16:16:16.381178 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.381144    2560 scope.go:117] "RemoveContainer" containerID="1fa0345c960e89c8b13e53b669ce5d52896c7122e5979e38e6cfd67e1ca4ec28"
Jul 29 16:16:16.389854 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.389818    2560 scope.go:117] "RemoveContainer" containerID="1fa0345c960e89c8b13e53b669ce5d52896c7122e5979e38e6cfd67e1ca4ec28"
Jul 29 16:16:16.390116 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:16:16.390098    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fa0345c960e89c8b13e53b669ce5d52896c7122e5979e38e6cfd67e1ca4ec28\": container with ID starting with 1fa0345c960e89c8b13e53b669ce5d52896c7122e5979e38e6cfd67e1ca4ec28 not found: ID does not exist" containerID="1fa0345c960e89c8b13e53b669ce5d52896c7122e5979e38e6cfd67e1ca4ec28"
Jul 29 16:16:16.390176 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.390124    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fa0345c960e89c8b13e53b669ce5d52896c7122e5979e38e6cfd67e1ca4ec28"} err="failed to get container status \"1fa0345c960e89c8b13e53b669ce5d52896c7122e5979e38e6cfd67e1ca4ec28\": rpc error: code = NotFound desc = could not find container \"1fa0345c960e89c8b13e53b669ce5d52896c7122e5979e38e6cfd67e1ca4ec28\": container with ID starting with 1fa0345c960e89c8b13e53b669ce5d52896c7122e5979e38e6cfd67e1ca4ec28 not found: ID does not exist"
Jul 29 16:16:16.430040 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.430009    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3413daf-1bed-4eb7-96e3-8590c49139d1-plugins-config-volume\") pod \"f3413daf-1bed-4eb7-96e3-8590c49139d1\" (UID: \"f3413daf-1bed-4eb7-96e3-8590c49139d1\") "
Jul 29 16:16:16.430205 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.430048    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf11ccb0-3442-4215-84ad-3084aa8326a1-plugins-config-volume\") pod \"cf11ccb0-3442-4215-84ad-3084aa8326a1\" (UID: \"cf11ccb0-3442-4215-84ad-3084aa8326a1\") "
Jul 29 16:16:16.430205 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.430080    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8b4t\" (UniqueName: \"kubernetes.io/projected/f3413daf-1bed-4eb7-96e3-8590c49139d1-kube-api-access-g8b4t\") pod \"f3413daf-1bed-4eb7-96e3-8590c49139d1\" (UID: \"f3413daf-1bed-4eb7-96e3-8590c49139d1\") "
Jul 29 16:16:16.430205 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.430116    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg7cl\" (UniqueName: \"kubernetes.io/projected/cf11ccb0-3442-4215-84ad-3084aa8326a1-kube-api-access-zg7cl\") pod \"cf11ccb0-3442-4215-84ad-3084aa8326a1\" (UID: \"cf11ccb0-3442-4215-84ad-3084aa8326a1\") "
Jul 29 16:16:16.430456 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.430434    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf11ccb0-3442-4215-84ad-3084aa8326a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf11ccb0-3442-4215-84ad-3084aa8326a1" (UID: "cf11ccb0-3442-4215-84ad-3084aa8326a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:16:16.430522 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.430438    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3413daf-1bed-4eb7-96e3-8590c49139d1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f3413daf-1bed-4eb7-96e3-8590c49139d1" (UID: "f3413daf-1bed-4eb7-96e3-8590c49139d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:16:16.432145 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.432123    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3413daf-1bed-4eb7-96e3-8590c49139d1-kube-api-access-g8b4t" (OuterVolumeSpecName: "kube-api-access-g8b4t") pod "f3413daf-1bed-4eb7-96e3-8590c49139d1" (UID: "f3413daf-1bed-4eb7-96e3-8590c49139d1"). InnerVolumeSpecName "kube-api-access-g8b4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:16:16.432214 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.432128    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf11ccb0-3442-4215-84ad-3084aa8326a1-kube-api-access-zg7cl" (OuterVolumeSpecName: "kube-api-access-zg7cl") pod "cf11ccb0-3442-4215-84ad-3084aa8326a1" (UID: "cf11ccb0-3442-4215-84ad-3084aa8326a1"). InnerVolumeSpecName "kube-api-access-zg7cl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:16:16.531155 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.531105    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3413daf-1bed-4eb7-96e3-8590c49139d1-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:16:16.531155 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.531148    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf11ccb0-3442-4215-84ad-3084aa8326a1-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:16:16.531155 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.531158    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8b4t\" (UniqueName: \"kubernetes.io/projected/f3413daf-1bed-4eb7-96e3-8590c49139d1-kube-api-access-g8b4t\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:16:16.531155 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.531168    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg7cl\" (UniqueName: \"kubernetes.io/projected/cf11ccb0-3442-4215-84ad-3084aa8326a1-kube-api-access-zg7cl\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:16:16.697217 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.697184    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"]
Jul 29 16:16:16.700556 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.700530    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-facc8562-65478d848b-85mf8"]
Jul 29 16:16:16.715374 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.715351    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"]
Jul 29 16:16:16.722569 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:16.722545    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-facc8562-797cf4685b-n52rn"]
Jul 29 16:16:18.042439 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:18.042408    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf11ccb0-3442-4215-84ad-3084aa8326a1" path="/var/lib/kubelet/pods/cf11ccb0-3442-4215-84ad-3084aa8326a1/volumes"
Jul 29 16:16:18.042802 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:18.042760    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3413daf-1bed-4eb7-96e3-8590c49139d1" path="/var/lib/kubelet/pods/f3413daf-1bed-4eb7-96e3-8590c49139d1/volumes"
Jul 29 16:16:20.259241 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:20.259205    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:16:31.473091 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:16:31.473012    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:17:41.950802 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:17:41.950762    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:17:54.979363 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:17:54.979325    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:18:06.663469 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:06.663391    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:18:11.928795 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:11.928761    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"]
Jul 29 16:18:11.929217 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:11.929106    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf11ccb0-3442-4215-84ad-3084aa8326a1" containerName="payload-processing"
Jul 29 16:18:11.929217 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:11.929117    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf11ccb0-3442-4215-84ad-3084aa8326a1" containerName="payload-processing"
Jul 29 16:18:11.929217 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:11.929138    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3413daf-1bed-4eb7-96e3-8590c49139d1" containerName="payload-pre-processing"
Jul 29 16:18:11.929217 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:11.929144    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3413daf-1bed-4eb7-96e3-8590c49139d1" containerName="payload-pre-processing"
Jul 29 16:18:11.929217 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:11.929200    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3413daf-1bed-4eb7-96e3-8590c49139d1" containerName="payload-pre-processing"
Jul 29 16:18:11.929217 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:11.929207    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf11ccb0-3442-4215-84ad-3084aa8326a1" containerName="payload-processing"
Jul 29 16:18:11.932115 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:11.932098    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"
Jul 29 16:18:11.934983 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:11.934964    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-82c422a4\""
Jul 29 16:18:11.935199 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:11.935184    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-82c422a4-dockercfg-4hcm8\""
Jul 29 16:18:11.946308 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:11.946282    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"]
Jul 29 16:18:12.018452 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.018421    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"]
Jul 29 16:18:12.019867 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.019846    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1008ed2-b7f5-4510-8432-f954441b3347-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g\" (UID: \"a1008ed2-b7f5-4510-8432-f954441b3347\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"
Jul 29 16:18:12.019965 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.019894    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-476z8\" (UniqueName: \"kubernetes.io/projected/a1008ed2-b7f5-4510-8432-f954441b3347-kube-api-access-476z8\") pod \"payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g\" (UID: \"a1008ed2-b7f5-4510-8432-f954441b3347\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"
Jul 29 16:18:12.021835 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.021820    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"
Jul 29 16:18:12.043869 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.043844    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"]
Jul 29 16:18:12.121057 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.121028    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-476z8\" (UniqueName: \"kubernetes.io/projected/a1008ed2-b7f5-4510-8432-f954441b3347-kube-api-access-476z8\") pod \"payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g\" (UID: \"a1008ed2-b7f5-4510-8432-f954441b3347\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"
Jul 29 16:18:12.121181 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.121071    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dctnn\" (UniqueName: \"kubernetes.io/projected/ae1159bb-cd9b-4aea-9414-a6fdd74c282d-kube-api-access-dctnn\") pod \"payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq\" (UID: \"ae1159bb-cd9b-4aea-9414-a6fdd74c282d\") " pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"
Jul 29 16:18:12.121181 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.121112    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae1159bb-cd9b-4aea-9414-a6fdd74c282d-plugins-config-volume\") pod \"payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq\" (UID: \"ae1159bb-cd9b-4aea-9414-a6fdd74c282d\") " pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"
Jul 29 16:18:12.121181 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.121151    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1008ed2-b7f5-4510-8432-f954441b3347-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g\" (UID: \"a1008ed2-b7f5-4510-8432-f954441b3347\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"
Jul 29 16:18:12.121765 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.121747    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1008ed2-b7f5-4510-8432-f954441b3347-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g\" (UID: \"a1008ed2-b7f5-4510-8432-f954441b3347\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"
Jul 29 16:18:12.142062 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.142036    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-476z8\" (UniqueName: \"kubernetes.io/projected/a1008ed2-b7f5-4510-8432-f954441b3347-kube-api-access-476z8\") pod \"payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g\" (UID: \"a1008ed2-b7f5-4510-8432-f954441b3347\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"
Jul 29 16:18:12.222573 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.222498    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dctnn\" (UniqueName: \"kubernetes.io/projected/ae1159bb-cd9b-4aea-9414-a6fdd74c282d-kube-api-access-dctnn\") pod \"payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq\" (UID: \"ae1159bb-cd9b-4aea-9414-a6fdd74c282d\") " pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"
Jul 29 16:18:12.222573 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.222544    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae1159bb-cd9b-4aea-9414-a6fdd74c282d-plugins-config-volume\") pod \"payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq\" (UID: \"ae1159bb-cd9b-4aea-9414-a6fdd74c282d\") " pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"
Jul 29 16:18:12.223214 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.223197    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae1159bb-cd9b-4aea-9414-a6fdd74c282d-plugins-config-volume\") pod \"payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq\" (UID: \"ae1159bb-cd9b-4aea-9414-a6fdd74c282d\") " pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"
Jul 29 16:18:12.232772 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.232745    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dctnn\" (UniqueName: \"kubernetes.io/projected/ae1159bb-cd9b-4aea-9414-a6fdd74c282d-kube-api-access-dctnn\") pod \"payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq\" (UID: \"ae1159bb-cd9b-4aea-9414-a6fdd74c282d\") " pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"
Jul 29 16:18:12.242504 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.242484    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"
Jul 29 16:18:12.331287 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.331252    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"
Jul 29 16:18:12.381994 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.381950    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"]
Jul 29 16:18:12.386520 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:18:12.386485    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1008ed2_b7f5_4510_8432_f954441b3347.slice/crio-b1c3ec620e01e3e02e6b3317f41c4c69d80e900754b3eae562627e1f4803dd05 WatchSource:0}: Error finding container b1c3ec620e01e3e02e6b3317f41c4c69d80e900754b3eae562627e1f4803dd05: Status 404 returned error can't find the container with id b1c3ec620e01e3e02e6b3317f41c4c69d80e900754b3eae562627e1f4803dd05
Jul 29 16:18:12.479377 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.479351    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"]
Jul 29 16:18:12.480453 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:18:12.480429    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae1159bb_cd9b_4aea_9414_a6fdd74c282d.slice/crio-351d7c773cceb97a182f46a36e3ab3a805084d3e4d6e39a3d4f7d15eb8908f6b WatchSource:0}: Error finding container 351d7c773cceb97a182f46a36e3ab3a805084d3e4d6e39a3d4f7d15eb8908f6b: Status 404 returned error can't find the container with id 351d7c773cceb97a182f46a36e3ab3a805084d3e4d6e39a3d4f7d15eb8908f6b
Jul 29 16:18:12.908563 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.908527    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq" event={"ID":"ae1159bb-cd9b-4aea-9414-a6fdd74c282d","Type":"ContainerStarted","Data":"1ea157ff8b4425d729d6695c7bb93261015af2c8b80c7b290fa1b6c08cc44f9e"}
Jul 29 16:18:12.908752 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.908578    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq" event={"ID":"ae1159bb-cd9b-4aea-9414-a6fdd74c282d","Type":"ContainerStarted","Data":"351d7c773cceb97a182f46a36e3ab3a805084d3e4d6e39a3d4f7d15eb8908f6b"}
Jul 29 16:18:12.908752 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.908616    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"
Jul 29 16:18:12.910210 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.910185    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g" event={"ID":"a1008ed2-b7f5-4510-8432-f954441b3347","Type":"ContainerStarted","Data":"e051ef61b169a4797d78465bbee77db4fa55fcb91784c1d13b6b1cb9108c24d5"}
Jul 29 16:18:12.910330 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.910228    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g" event={"ID":"a1008ed2-b7f5-4510-8432-f954441b3347","Type":"ContainerStarted","Data":"b1c3ec620e01e3e02e6b3317f41c4c69d80e900754b3eae562627e1f4803dd05"}
Jul 29 16:18:12.910330 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.910248    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"
Jul 29 16:18:12.933391 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.933351    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq" podStartSLOduration=1.9333384740000001 podStartE2EDuration="1.933338474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:18:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:18:12.933015381 +0000 UTC m=+1897.375221640" watchObservedRunningTime="2026-07-29 16:18:12.933338474 +0000 UTC m=+1897.375544733"
Jul 29 16:18:12.960343 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:12.960296    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g" podStartSLOduration=1.960283078 podStartE2EDuration="1.960283078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:18:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:18:12.959327426 +0000 UTC m=+1897.401533685" watchObservedRunningTime="2026-07-29 16:18:12.960283078 +0000 UTC m=+1897.402489338"
Jul 29 16:18:17.264778 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:17.264684    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:18:23.920091 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:23.920057    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"
Jul 29 16:18:23.920516 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:23.920135    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"
Jul 29 16:18:54.381972 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.381938    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"]
Jul 29 16:18:54.382538 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.382195    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq" podUID="ae1159bb-cd9b-4aea-9414-a6fdd74c282d" containerName="payload-processing" containerID="cri-o://1ea157ff8b4425d729d6695c7bb93261015af2c8b80c7b290fa1b6c08cc44f9e" gracePeriod=30
Jul 29 16:18:54.408705 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.408678    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"]
Jul 29 16:18:54.408889 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.408855    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g" podUID="a1008ed2-b7f5-4510-8432-f954441b3347" containerName="payload-pre-processing" containerID="cri-o://e051ef61b169a4797d78465bbee77db4fa55fcb91784c1d13b6b1cb9108c24d5" gracePeriod=30
Jul 29 16:18:54.674603 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.674582    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"
Jul 29 16:18:54.677617 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.677600    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"
Jul 29 16:18:54.778646 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.778613    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1008ed2-b7f5-4510-8432-f954441b3347-plugins-config-volume\") pod \"a1008ed2-b7f5-4510-8432-f954441b3347\" (UID: \"a1008ed2-b7f5-4510-8432-f954441b3347\") "
Jul 29 16:18:54.778820 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.778673    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae1159bb-cd9b-4aea-9414-a6fdd74c282d-plugins-config-volume\") pod \"ae1159bb-cd9b-4aea-9414-a6fdd74c282d\" (UID: \"ae1159bb-cd9b-4aea-9414-a6fdd74c282d\") "
Jul 29 16:18:54.778820 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.778730    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dctnn\" (UniqueName: \"kubernetes.io/projected/ae1159bb-cd9b-4aea-9414-a6fdd74c282d-kube-api-access-dctnn\") pod \"ae1159bb-cd9b-4aea-9414-a6fdd74c282d\" (UID: \"ae1159bb-cd9b-4aea-9414-a6fdd74c282d\") "
Jul 29 16:18:54.778820 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.778804    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-476z8\" (UniqueName: \"kubernetes.io/projected/a1008ed2-b7f5-4510-8432-f954441b3347-kube-api-access-476z8\") pod \"a1008ed2-b7f5-4510-8432-f954441b3347\" (UID: \"a1008ed2-b7f5-4510-8432-f954441b3347\") "
Jul 29 16:18:54.779080 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.779056    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1008ed2-b7f5-4510-8432-f954441b3347-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a1008ed2-b7f5-4510-8432-f954441b3347" (UID: "a1008ed2-b7f5-4510-8432-f954441b3347"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:18:54.779080 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.779062    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae1159bb-cd9b-4aea-9414-a6fdd74c282d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ae1159bb-cd9b-4aea-9414-a6fdd74c282d" (UID: "ae1159bb-cd9b-4aea-9414-a6fdd74c282d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:18:54.780772 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.780746    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae1159bb-cd9b-4aea-9414-a6fdd74c282d-kube-api-access-dctnn" (OuterVolumeSpecName: "kube-api-access-dctnn") pod "ae1159bb-cd9b-4aea-9414-a6fdd74c282d" (UID: "ae1159bb-cd9b-4aea-9414-a6fdd74c282d"). InnerVolumeSpecName "kube-api-access-dctnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:18:54.780886 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.780801    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1008ed2-b7f5-4510-8432-f954441b3347-kube-api-access-476z8" (OuterVolumeSpecName: "kube-api-access-476z8") pod "a1008ed2-b7f5-4510-8432-f954441b3347" (UID: "a1008ed2-b7f5-4510-8432-f954441b3347"). InnerVolumeSpecName "kube-api-access-476z8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:18:54.880122 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.880092    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1008ed2-b7f5-4510-8432-f954441b3347-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:18:54.880122 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.880115    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae1159bb-cd9b-4aea-9414-a6fdd74c282d-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:18:54.880122 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.880124    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dctnn\" (UniqueName: \"kubernetes.io/projected/ae1159bb-cd9b-4aea-9414-a6fdd74c282d-kube-api-access-dctnn\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:18:54.880122 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:54.880132    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-476z8\" (UniqueName: \"kubernetes.io/projected/a1008ed2-b7f5-4510-8432-f954441b3347-kube-api-access-476z8\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:18:55.110598 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.110507    2560 generic.go:356] "Generic (PLEG): container finished" podID="ae1159bb-cd9b-4aea-9414-a6fdd74c282d" containerID="1ea157ff8b4425d729d6695c7bb93261015af2c8b80c7b290fa1b6c08cc44f9e" exitCode=0
Jul 29 16:18:55.110598 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.110555    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"
Jul 29 16:18:55.110598 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.110560    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq" event={"ID":"ae1159bb-cd9b-4aea-9414-a6fdd74c282d","Type":"ContainerDied","Data":"1ea157ff8b4425d729d6695c7bb93261015af2c8b80c7b290fa1b6c08cc44f9e"}
Jul 29 16:18:55.110948 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.110615    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq" event={"ID":"ae1159bb-cd9b-4aea-9414-a6fdd74c282d","Type":"ContainerDied","Data":"351d7c773cceb97a182f46a36e3ab3a805084d3e4d6e39a3d4f7d15eb8908f6b"}
Jul 29 16:18:55.110948 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.110650    2560 scope.go:117] "RemoveContainer" containerID="1ea157ff8b4425d729d6695c7bb93261015af2c8b80c7b290fa1b6c08cc44f9e"
Jul 29 16:18:55.111996 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.111972    2560 generic.go:356] "Generic (PLEG): container finished" podID="a1008ed2-b7f5-4510-8432-f954441b3347" containerID="e051ef61b169a4797d78465bbee77db4fa55fcb91784c1d13b6b1cb9108c24d5" exitCode=0
Jul 29 16:18:55.112135 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.112026    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"
Jul 29 16:18:55.112135 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.112072    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g" event={"ID":"a1008ed2-b7f5-4510-8432-f954441b3347","Type":"ContainerDied","Data":"e051ef61b169a4797d78465bbee77db4fa55fcb91784c1d13b6b1cb9108c24d5"}
Jul 29 16:18:55.112259 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.112134    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g" event={"ID":"a1008ed2-b7f5-4510-8432-f954441b3347","Type":"ContainerDied","Data":"b1c3ec620e01e3e02e6b3317f41c4c69d80e900754b3eae562627e1f4803dd05"}
Jul 29 16:18:55.119096 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.119077    2560 scope.go:117] "RemoveContainer" containerID="1ea157ff8b4425d729d6695c7bb93261015af2c8b80c7b290fa1b6c08cc44f9e"
Jul 29 16:18:55.119336 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:18:55.119316    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ea157ff8b4425d729d6695c7bb93261015af2c8b80c7b290fa1b6c08cc44f9e\": container with ID starting with 1ea157ff8b4425d729d6695c7bb93261015af2c8b80c7b290fa1b6c08cc44f9e not found: ID does not exist" containerID="1ea157ff8b4425d729d6695c7bb93261015af2c8b80c7b290fa1b6c08cc44f9e"
Jul 29 16:18:55.119383 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.119346    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ea157ff8b4425d729d6695c7bb93261015af2c8b80c7b290fa1b6c08cc44f9e"} err="failed to get container status \"1ea157ff8b4425d729d6695c7bb93261015af2c8b80c7b290fa1b6c08cc44f9e\": rpc error: code = NotFound desc = could not find container \"1ea157ff8b4425d729d6695c7bb93261015af2c8b80c7b290fa1b6c08cc44f9e\": container with ID starting with 1ea157ff8b4425d729d6695c7bb93261015af2c8b80c7b290fa1b6c08cc44f9e not found: ID does not exist"
Jul 29 16:18:55.119383 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.119363    2560 scope.go:117] "RemoveContainer" containerID="e051ef61b169a4797d78465bbee77db4fa55fcb91784c1d13b6b1cb9108c24d5"
Jul 29 16:18:55.127107 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.127090    2560 scope.go:117] "RemoveContainer" containerID="e051ef61b169a4797d78465bbee77db4fa55fcb91784c1d13b6b1cb9108c24d5"
Jul 29 16:18:55.127337 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:18:55.127317    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e051ef61b169a4797d78465bbee77db4fa55fcb91784c1d13b6b1cb9108c24d5\": container with ID starting with e051ef61b169a4797d78465bbee77db4fa55fcb91784c1d13b6b1cb9108c24d5 not found: ID does not exist" containerID="e051ef61b169a4797d78465bbee77db4fa55fcb91784c1d13b6b1cb9108c24d5"
Jul 29 16:18:55.127384 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.127344    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e051ef61b169a4797d78465bbee77db4fa55fcb91784c1d13b6b1cb9108c24d5"} err="failed to get container status \"e051ef61b169a4797d78465bbee77db4fa55fcb91784c1d13b6b1cb9108c24d5\": rpc error: code = NotFound desc = could not find container \"e051ef61b169a4797d78465bbee77db4fa55fcb91784c1d13b6b1cb9108c24d5\": container with ID starting with e051ef61b169a4797d78465bbee77db4fa55fcb91784c1d13b6b1cb9108c24d5 not found: ID does not exist"
Jul 29 16:18:55.145845 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.145824    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"]
Jul 29 16:18:55.149295 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.149276    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-82c422a4-97599f99d-hzwvq"]
Jul 29 16:18:55.166604 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.166584    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"]
Jul 29 16:18:55.168732 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:55.168713    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-82c422a4-56bb89d4cb-59t2g"]
Jul 29 16:18:56.043238 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:56.043207    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1008ed2-b7f5-4510-8432-f954441b3347" path="/var/lib/kubelet/pods/a1008ed2-b7f5-4510-8432-f954441b3347/volumes"
Jul 29 16:18:56.043587 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:56.043553    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae1159bb-cd9b-4aea-9414-a6fdd74c282d" path="/var/lib/kubelet/pods/ae1159bb-cd9b-4aea-9414-a6fdd74c282d/volumes"
Jul 29 16:18:57.456142 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:18:57.456112    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:19:28.718073 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.717989    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"]
Jul 29 16:19:28.718531 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.718327    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1008ed2-b7f5-4510-8432-f954441b3347" containerName="payload-pre-processing"
Jul 29 16:19:28.718531 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.718338    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1008ed2-b7f5-4510-8432-f954441b3347" containerName="payload-pre-processing"
Jul 29 16:19:28.718531 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.718350    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1159bb-cd9b-4aea-9414-a6fdd74c282d" containerName="payload-processing"
Jul 29 16:19:28.718531 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.718356    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1159bb-cd9b-4aea-9414-a6fdd74c282d" containerName="payload-processing"
Jul 29 16:19:28.718531 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.718422    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1008ed2-b7f5-4510-8432-f954441b3347" containerName="payload-pre-processing"
Jul 29 16:19:28.718531 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.718431    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae1159bb-cd9b-4aea-9414-a6fdd74c282d" containerName="payload-processing"
Jul 29 16:19:28.721155 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.721137    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"
Jul 29 16:19:28.724131 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.724109    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c6a1f06a\""
Jul 29 16:19:28.724263 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.724196    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c6a1f06a-dockercfg-pj42w\""
Jul 29 16:19:28.730901 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.730876    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"]
Jul 29 16:19:28.748509 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.748486    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"]
Jul 29 16:19:28.751995 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.751978    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"
Jul 29 16:19:28.762423 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.762405    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"]
Jul 29 16:19:28.764063 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.764038    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e302cd3-a8a7-443e-8c6d-338dca091eb7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz\" (UID: \"1e302cd3-a8a7-443e-8c6d-338dca091eb7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"
Jul 29 16:19:28.764158 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.764090    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82c56c4-d080-42b4-b95e-d7e7449dc36c-plugins-config-volume\") pod \"payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh\" (UID: \"b82c56c4-d080-42b4-b95e-d7e7449dc36c\") " pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"
Jul 29 16:19:28.764158 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.764143    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zjxh\" (UniqueName: \"kubernetes.io/projected/b82c56c4-d080-42b4-b95e-d7e7449dc36c-kube-api-access-7zjxh\") pod \"payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh\" (UID: \"b82c56c4-d080-42b4-b95e-d7e7449dc36c\") " pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"
Jul 29 16:19:28.764239 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.764174    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq74c\" (UniqueName: \"kubernetes.io/projected/1e302cd3-a8a7-443e-8c6d-338dca091eb7-kube-api-access-kq74c\") pod \"payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz\" (UID: \"1e302cd3-a8a7-443e-8c6d-338dca091eb7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"
Jul 29 16:19:28.864577 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.864540    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e302cd3-a8a7-443e-8c6d-338dca091eb7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz\" (UID: \"1e302cd3-a8a7-443e-8c6d-338dca091eb7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"
Jul 29 16:19:28.864772 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.864585    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82c56c4-d080-42b4-b95e-d7e7449dc36c-plugins-config-volume\") pod \"payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh\" (UID: \"b82c56c4-d080-42b4-b95e-d7e7449dc36c\") " pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"
Jul 29 16:19:28.864772 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.864609    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zjxh\" (UniqueName: \"kubernetes.io/projected/b82c56c4-d080-42b4-b95e-d7e7449dc36c-kube-api-access-7zjxh\") pod \"payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh\" (UID: \"b82c56c4-d080-42b4-b95e-d7e7449dc36c\") " pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"
Jul 29 16:19:28.864772 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.864628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq74c\" (UniqueName: \"kubernetes.io/projected/1e302cd3-a8a7-443e-8c6d-338dca091eb7-kube-api-access-kq74c\") pod \"payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz\" (UID: \"1e302cd3-a8a7-443e-8c6d-338dca091eb7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"
Jul 29 16:19:28.865543 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.865517    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82c56c4-d080-42b4-b95e-d7e7449dc36c-plugins-config-volume\") pod \"payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh\" (UID: \"b82c56c4-d080-42b4-b95e-d7e7449dc36c\") " pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"
Jul 29 16:19:28.865598 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.865517    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e302cd3-a8a7-443e-8c6d-338dca091eb7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz\" (UID: \"1e302cd3-a8a7-443e-8c6d-338dca091eb7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"
Jul 29 16:19:28.873289 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.873262    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zjxh\" (UniqueName: \"kubernetes.io/projected/b82c56c4-d080-42b4-b95e-d7e7449dc36c-kube-api-access-7zjxh\") pod \"payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh\" (UID: \"b82c56c4-d080-42b4-b95e-d7e7449dc36c\") " pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"
Jul 29 16:19:28.873386 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:28.873300    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq74c\" (UniqueName: \"kubernetes.io/projected/1e302cd3-a8a7-443e-8c6d-338dca091eb7-kube-api-access-kq74c\") pod \"payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz\" (UID: \"1e302cd3-a8a7-443e-8c6d-338dca091eb7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"
Jul 29 16:19:29.032624 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:29.032536    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"
Jul 29 16:19:29.061316 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:29.061283    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"
Jul 29 16:19:29.183556 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:29.183529    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"]
Jul 29 16:19:29.185648 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:19:29.185623    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb82c56c4_d080_42b4_b95e_d7e7449dc36c.slice/crio-f8933a605d65587bc8a6cf577da1de6de602629f96221edafd9c547316a46d03 WatchSource:0}: Error finding container f8933a605d65587bc8a6cf577da1de6de602629f96221edafd9c547316a46d03: Status 404 returned error can't find the container with id f8933a605d65587bc8a6cf577da1de6de602629f96221edafd9c547316a46d03
Jul 29 16:19:29.271208 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:29.271173    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh" event={"ID":"b82c56c4-d080-42b4-b95e-d7e7449dc36c","Type":"ContainerStarted","Data":"f69c690fe33edf8c4abf12961190d3354d1502055e21afc8d65686bcf5cd888e"}
Jul 29 16:19:29.271355 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:29.271235    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh" event={"ID":"b82c56c4-d080-42b4-b95e-d7e7449dc36c","Type":"ContainerStarted","Data":"f8933a605d65587bc8a6cf577da1de6de602629f96221edafd9c547316a46d03"}
Jul 29 16:19:29.271398 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:29.271349    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"
Jul 29 16:19:29.290759 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:29.290671    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh" podStartSLOduration=1.290658365 podStartE2EDuration="1.290658365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:19:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:19:29.289265979 +0000 UTC m=+1973.731472237" watchObservedRunningTime="2026-07-29 16:19:29.290658365 +0000 UTC m=+1973.732864624"
Jul 29 16:19:29.412827 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:29.412800    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"]
Jul 29 16:19:29.414707 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:19:29.414682    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e302cd3_a8a7_443e_8c6d_338dca091eb7.slice/crio-12e7c223f81bb0d8a96aaf25aadcc6a855ba5691b327b52b6ead5119a383acaf WatchSource:0}: Error finding container 12e7c223f81bb0d8a96aaf25aadcc6a855ba5691b327b52b6ead5119a383acaf: Status 404 returned error can't find the container with id 12e7c223f81bb0d8a96aaf25aadcc6a855ba5691b327b52b6ead5119a383acaf
Jul 29 16:19:30.277332 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:30.277297    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz" event={"ID":"1e302cd3-a8a7-443e-8c6d-338dca091eb7","Type":"ContainerStarted","Data":"f2b5b739bb70c8c592effb3a7690b8c3fe98fb4e99f009ec169eca91fcdd7414"}
Jul 29 16:19:30.277749 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:30.277432    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"
Jul 29 16:19:30.277749 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:30.277457    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz" event={"ID":"1e302cd3-a8a7-443e-8c6d-338dca091eb7","Type":"ContainerStarted","Data":"12e7c223f81bb0d8a96aaf25aadcc6a855ba5691b327b52b6ead5119a383acaf"}
Jul 29 16:19:30.294858 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:30.294810    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz" podStartSLOduration=2.294795358 podStartE2EDuration="2.294795358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:19:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:19:30.294363782 +0000 UTC m=+1974.736570042" watchObservedRunningTime="2026-07-29 16:19:30.294795358 +0000 UTC m=+1974.737001617"
Jul 29 16:19:34.159836 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:34.159803    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:19:40.279683 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:40.279650    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"
Jul 29 16:19:41.285101 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:19:41.285073    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"
Jul 29 16:20:02.349850 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.349809    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"]
Jul 29 16:20:02.354118 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.354092    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.357358 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.357329    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c6a1f06a\"/\"openshift-service-ca.crt\""
Jul 29 16:20:02.358529 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.358504    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c6a1f06a\"/\"e2e-lifecycle-model-c6a1f06a-kserve-self-signed-certs\""
Jul 29 16:20:02.358666 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.358607    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c6a1f06a\"/\"kube-root-ca.crt\""
Jul 29 16:20:02.358666 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.358619    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c6a1f06a\"/\"default-dockercfg-q8z2s\""
Jul 29 16:20:02.364960 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.364932    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"]
Jul 29 16:20:02.420590 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.420540    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-model-cache\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.420780 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.420605    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-home\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.420780 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.420637    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509892d1-d707-4ce4-936a-b44c374c3e29-tls-certs\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.420780 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.420668    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjmvq\" (UniqueName: \"kubernetes.io/projected/509892d1-d707-4ce4-936a-b44c374c3e29-kube-api-access-rjmvq\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.420780 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.420711    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-tmp-dir\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.420780 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.420760    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-dshm\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.521643 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.521592    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-model-cache\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.521643 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.521658    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-home\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.521868 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.521753    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509892d1-d707-4ce4-936a-b44c374c3e29-tls-certs\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.521868 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.521791    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjmvq\" (UniqueName: \"kubernetes.io/projected/509892d1-d707-4ce4-936a-b44c374c3e29-kube-api-access-rjmvq\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.521868 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.521850    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-tmp-dir\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.522008 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.521892    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-dshm\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.522078 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.522041    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-model-cache\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.522137 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.522067    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-home\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.522242 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.522221    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-tmp-dir\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.524126 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.524103    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-dshm\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.524421 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.524402    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509892d1-d707-4ce4-936a-b44c374c3e29-tls-certs\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.530635 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.530613    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjmvq\" (UniqueName: \"kubernetes.io/projected/509892d1-d707-4ce4-936a-b44c374c3e29-kube-api-access-rjmvq\") pod \"e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") " pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.664612 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.664579    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:02.803880 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:02.803855    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"]
Jul 29 16:20:02.805808 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:20:02.805780    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod509892d1_d707_4ce4_936a_b44c374c3e29.slice/crio-f1388c6a7d149dd674f739948d639c04d34156d9f66767dc2b3dad496bac8f13 WatchSource:0}: Error finding container f1388c6a7d149dd674f739948d639c04d34156d9f66767dc2b3dad496bac8f13: Status 404 returned error can't find the container with id f1388c6a7d149dd674f739948d639c04d34156d9f66767dc2b3dad496bac8f13
Jul 29 16:20:03.434959 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:03.434899    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt" event={"ID":"509892d1-d707-4ce4-936a-b44c374c3e29","Type":"ContainerStarted","Data":"189177b29180daba2eb513f17a00a4c2987ac9e0b4627efb7580d75712312d87"}
Jul 29 16:20:03.435345 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:03.434966    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt" event={"ID":"509892d1-d707-4ce4-936a-b44c374c3e29","Type":"ContainerStarted","Data":"f1388c6a7d149dd674f739948d639c04d34156d9f66767dc2b3dad496bac8f13"}
Jul 29 16:20:03.454770 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:03.454715    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt" podStartSLOduration=1.454699199 podStartE2EDuration="1.454699199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:20:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:20:03.453200698 +0000 UTC m=+2007.895406948" watchObservedRunningTime="2026-07-29 16:20:03.454699199 +0000 UTC m=+2007.896905457"
Jul 29 16:20:05.657905 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:05.657868    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:20:12.665403 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:12.665363    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:12.665899 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:12.665426    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:12.678260 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:12.678235    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:13.492125 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:13.492099    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:20:24.950717 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:24.950678    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:20:46.449851 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:46.449818    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:20:47.825959 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:47.825925    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"]
Jul 29 16:20:47.826391 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:47.826148    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh" podUID="b82c56c4-d080-42b4-b95e-d7e7449dc36c" containerName="payload-processing" containerID="cri-o://f69c690fe33edf8c4abf12961190d3354d1502055e21afc8d65686bcf5cd888e" gracePeriod=30
Jul 29 16:20:47.833041 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:47.833014    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"]
Jul 29 16:20:47.833214 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:47.833194    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz" podUID="1e302cd3-a8a7-443e-8c6d-338dca091eb7" containerName="payload-pre-processing" containerID="cri-o://f2b5b739bb70c8c592effb3a7690b8c3fe98fb4e99f009ec169eca91fcdd7414" gracePeriod=30
Jul 29 16:20:48.090554 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.090500    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"
Jul 29 16:20:48.115648 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.115617    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zjxh\" (UniqueName: \"kubernetes.io/projected/b82c56c4-d080-42b4-b95e-d7e7449dc36c-kube-api-access-7zjxh\") pod \"b82c56c4-d080-42b4-b95e-d7e7449dc36c\" (UID: \"b82c56c4-d080-42b4-b95e-d7e7449dc36c\") "
Jul 29 16:20:48.115763 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.115710    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82c56c4-d080-42b4-b95e-d7e7449dc36c-plugins-config-volume\") pod \"b82c56c4-d080-42b4-b95e-d7e7449dc36c\" (UID: \"b82c56c4-d080-42b4-b95e-d7e7449dc36c\") "
Jul 29 16:20:48.116040 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.116019    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b82c56c4-d080-42b4-b95e-d7e7449dc36c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b82c56c4-d080-42b4-b95e-d7e7449dc36c" (UID: "b82c56c4-d080-42b4-b95e-d7e7449dc36c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:20:48.116448 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.116435    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"
Jul 29 16:20:48.117614 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.117590    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b82c56c4-d080-42b4-b95e-d7e7449dc36c-kube-api-access-7zjxh" (OuterVolumeSpecName: "kube-api-access-7zjxh") pod "b82c56c4-d080-42b4-b95e-d7e7449dc36c" (UID: "b82c56c4-d080-42b4-b95e-d7e7449dc36c"). InnerVolumeSpecName "kube-api-access-7zjxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:20:48.216467 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.216427    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kq74c\" (UniqueName: \"kubernetes.io/projected/1e302cd3-a8a7-443e-8c6d-338dca091eb7-kube-api-access-kq74c\") pod \"1e302cd3-a8a7-443e-8c6d-338dca091eb7\" (UID: \"1e302cd3-a8a7-443e-8c6d-338dca091eb7\") "
Jul 29 16:20:48.216467 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.216472    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e302cd3-a8a7-443e-8c6d-338dca091eb7-plugins-config-volume\") pod \"1e302cd3-a8a7-443e-8c6d-338dca091eb7\" (UID: \"1e302cd3-a8a7-443e-8c6d-338dca091eb7\") "
Jul 29 16:20:48.216673 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.216625    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zjxh\" (UniqueName: \"kubernetes.io/projected/b82c56c4-d080-42b4-b95e-d7e7449dc36c-kube-api-access-7zjxh\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:20:48.216673 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.216637    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82c56c4-d080-42b4-b95e-d7e7449dc36c-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:20:48.216840 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.216817    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e302cd3-a8a7-443e-8c6d-338dca091eb7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1e302cd3-a8a7-443e-8c6d-338dca091eb7" (UID: "1e302cd3-a8a7-443e-8c6d-338dca091eb7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:20:48.218551 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.218529    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e302cd3-a8a7-443e-8c6d-338dca091eb7-kube-api-access-kq74c" (OuterVolumeSpecName: "kube-api-access-kq74c") pod "1e302cd3-a8a7-443e-8c6d-338dca091eb7" (UID: "1e302cd3-a8a7-443e-8c6d-338dca091eb7"). InnerVolumeSpecName "kube-api-access-kq74c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:20:48.317842 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.317809    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kq74c\" (UniqueName: \"kubernetes.io/projected/1e302cd3-a8a7-443e-8c6d-338dca091eb7-kube-api-access-kq74c\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:20:48.317842 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.317836    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e302cd3-a8a7-443e-8c6d-338dca091eb7-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:20:48.650094 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.650059    2560 generic.go:356] "Generic (PLEG): container finished" podID="1e302cd3-a8a7-443e-8c6d-338dca091eb7" containerID="f2b5b739bb70c8c592effb3a7690b8c3fe98fb4e99f009ec169eca91fcdd7414" exitCode=0
Jul 29 16:20:48.650282 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.650105    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"
Jul 29 16:20:48.650282 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.650145    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz" event={"ID":"1e302cd3-a8a7-443e-8c6d-338dca091eb7","Type":"ContainerDied","Data":"f2b5b739bb70c8c592effb3a7690b8c3fe98fb4e99f009ec169eca91fcdd7414"}
Jul 29 16:20:48.650282 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.650203    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz" event={"ID":"1e302cd3-a8a7-443e-8c6d-338dca091eb7","Type":"ContainerDied","Data":"12e7c223f81bb0d8a96aaf25aadcc6a855ba5691b327b52b6ead5119a383acaf"}
Jul 29 16:20:48.650282 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.650236    2560 scope.go:117] "RemoveContainer" containerID="f2b5b739bb70c8c592effb3a7690b8c3fe98fb4e99f009ec169eca91fcdd7414"
Jul 29 16:20:48.651493 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.651475    2560 generic.go:356] "Generic (PLEG): container finished" podID="b82c56c4-d080-42b4-b95e-d7e7449dc36c" containerID="f69c690fe33edf8c4abf12961190d3354d1502055e21afc8d65686bcf5cd888e" exitCode=0
Jul 29 16:20:48.651574 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.651504    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh" event={"ID":"b82c56c4-d080-42b4-b95e-d7e7449dc36c","Type":"ContainerDied","Data":"f69c690fe33edf8c4abf12961190d3354d1502055e21afc8d65686bcf5cd888e"}
Jul 29 16:20:48.651574 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.651526    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"
Jul 29 16:20:48.651574 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.651552    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh" event={"ID":"b82c56c4-d080-42b4-b95e-d7e7449dc36c","Type":"ContainerDied","Data":"f8933a605d65587bc8a6cf577da1de6de602629f96221edafd9c547316a46d03"}
Jul 29 16:20:48.659505 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.659486    2560 scope.go:117] "RemoveContainer" containerID="f2b5b739bb70c8c592effb3a7690b8c3fe98fb4e99f009ec169eca91fcdd7414"
Jul 29 16:20:48.659887 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:20:48.659784    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2b5b739bb70c8c592effb3a7690b8c3fe98fb4e99f009ec169eca91fcdd7414\": container with ID starting with f2b5b739bb70c8c592effb3a7690b8c3fe98fb4e99f009ec169eca91fcdd7414 not found: ID does not exist" containerID="f2b5b739bb70c8c592effb3a7690b8c3fe98fb4e99f009ec169eca91fcdd7414"
Jul 29 16:20:48.659887 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.659807    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2b5b739bb70c8c592effb3a7690b8c3fe98fb4e99f009ec169eca91fcdd7414"} err="failed to get container status \"f2b5b739bb70c8c592effb3a7690b8c3fe98fb4e99f009ec169eca91fcdd7414\": rpc error: code = NotFound desc = could not find container \"f2b5b739bb70c8c592effb3a7690b8c3fe98fb4e99f009ec169eca91fcdd7414\": container with ID starting with f2b5b739bb70c8c592effb3a7690b8c3fe98fb4e99f009ec169eca91fcdd7414 not found: ID does not exist"
Jul 29 16:20:48.659887 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.659823    2560 scope.go:117] "RemoveContainer" containerID="f69c690fe33edf8c4abf12961190d3354d1502055e21afc8d65686bcf5cd888e"
Jul 29 16:20:48.667623 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.667610    2560 scope.go:117] "RemoveContainer" containerID="f69c690fe33edf8c4abf12961190d3354d1502055e21afc8d65686bcf5cd888e"
Jul 29 16:20:48.667844 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:20:48.667828    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f69c690fe33edf8c4abf12961190d3354d1502055e21afc8d65686bcf5cd888e\": container with ID starting with f69c690fe33edf8c4abf12961190d3354d1502055e21afc8d65686bcf5cd888e not found: ID does not exist" containerID="f69c690fe33edf8c4abf12961190d3354d1502055e21afc8d65686bcf5cd888e"
Jul 29 16:20:48.667895 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.667849    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69c690fe33edf8c4abf12961190d3354d1502055e21afc8d65686bcf5cd888e"} err="failed to get container status \"f69c690fe33edf8c4abf12961190d3354d1502055e21afc8d65686bcf5cd888e\": rpc error: code = NotFound desc = could not find container \"f69c690fe33edf8c4abf12961190d3354d1502055e21afc8d65686bcf5cd888e\": container with ID starting with f69c690fe33edf8c4abf12961190d3354d1502055e21afc8d65686bcf5cd888e not found: ID does not exist"
Jul 29 16:20:48.675377 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.675351    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"]
Jul 29 16:20:48.679664 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.679646    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6a1f06a-b5cd7dcc8-q68rh"]
Jul 29 16:20:48.689597 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.689577    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"]
Jul 29 16:20:48.691238 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:48.691221    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6a1f06a-c66b5f66d-kdpcz"]
Jul 29 16:20:50.044016 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:50.043976    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e302cd3-a8a7-443e-8c6d-338dca091eb7" path="/var/lib/kubelet/pods/1e302cd3-a8a7-443e-8c6d-338dca091eb7/volumes"
Jul 29 16:20:50.044512 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:50.044476    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b82c56c4-d080-42b4-b95e-d7e7449dc36c" path="/var/lib/kubelet/pods/b82c56c4-d080-42b4-b95e-d7e7449dc36c/volumes"
Jul 29 16:20:56.164674 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:56.164641    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"]
Jul 29 16:20:56.946136 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:20:56.946101    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:21:11.183617 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.183527    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt" podUID="509892d1-d707-4ce4-936a-b44c374c3e29" containerName="main" containerID="cri-o://189177b29180daba2eb513f17a00a4c2987ac9e0b4627efb7580d75712312d87" gracePeriod=45
Jul 29 16:21:11.432239 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.432220    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:21:11.512361 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.512276    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-home\") pod \"509892d1-d707-4ce4-936a-b44c374c3e29\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") "
Jul 29 16:21:11.512361 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.512315    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509892d1-d707-4ce4-936a-b44c374c3e29-tls-certs\") pod \"509892d1-d707-4ce4-936a-b44c374c3e29\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") "
Jul 29 16:21:11.512361 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.512353    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjmvq\" (UniqueName: \"kubernetes.io/projected/509892d1-d707-4ce4-936a-b44c374c3e29-kube-api-access-rjmvq\") pod \"509892d1-d707-4ce4-936a-b44c374c3e29\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") "
Jul 29 16:21:11.512635 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.512411    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-tmp-dir\") pod \"509892d1-d707-4ce4-936a-b44c374c3e29\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") "
Jul 29 16:21:11.512635 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.512434    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-dshm\") pod \"509892d1-d707-4ce4-936a-b44c374c3e29\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") "
Jul 29 16:21:11.512635 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.512463    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-model-cache\") pod \"509892d1-d707-4ce4-936a-b44c374c3e29\" (UID: \"509892d1-d707-4ce4-936a-b44c374c3e29\") "
Jul 29 16:21:11.512635 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.512556    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-home" (OuterVolumeSpecName: "home") pod "509892d1-d707-4ce4-936a-b44c374c3e29" (UID: "509892d1-d707-4ce4-936a-b44c374c3e29"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:21:11.512821 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.512641    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "509892d1-d707-4ce4-936a-b44c374c3e29" (UID: "509892d1-d707-4ce4-936a-b44c374c3e29"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:21:11.512821 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.512703    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-home\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:21:11.512821 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.512719    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-tmp-dir\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:21:11.512821 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.512772    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-model-cache" (OuterVolumeSpecName: "model-cache") pod "509892d1-d707-4ce4-936a-b44c374c3e29" (UID: "509892d1-d707-4ce4-936a-b44c374c3e29"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:21:11.514639 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.514612    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-dshm" (OuterVolumeSpecName: "dshm") pod "509892d1-d707-4ce4-936a-b44c374c3e29" (UID: "509892d1-d707-4ce4-936a-b44c374c3e29"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:21:11.514725 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.514649    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/509892d1-d707-4ce4-936a-b44c374c3e29-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "509892d1-d707-4ce4-936a-b44c374c3e29" (UID: "509892d1-d707-4ce4-936a-b44c374c3e29"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:21:11.514725 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.514694    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/509892d1-d707-4ce4-936a-b44c374c3e29-kube-api-access-rjmvq" (OuterVolumeSpecName: "kube-api-access-rjmvq") pod "509892d1-d707-4ce4-936a-b44c374c3e29" (UID: "509892d1-d707-4ce4-936a-b44c374c3e29"). InnerVolumeSpecName "kube-api-access-rjmvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:21:11.614099 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.614065    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509892d1-d707-4ce4-936a-b44c374c3e29-tls-certs\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:21:11.614099 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.614094    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjmvq\" (UniqueName: \"kubernetes.io/projected/509892d1-d707-4ce4-936a-b44c374c3e29-kube-api-access-rjmvq\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:21:11.614099 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.614107    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-dshm\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:21:11.614321 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.614120    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509892d1-d707-4ce4-936a-b44c374c3e29-model-cache\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:21:11.765463 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.765373    2560 generic.go:356] "Generic (PLEG): container finished" podID="509892d1-d707-4ce4-936a-b44c374c3e29" containerID="189177b29180daba2eb513f17a00a4c2987ac9e0b4627efb7580d75712312d87" exitCode=0
Jul 29 16:21:11.765463 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.765426    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"
Jul 29 16:21:11.765651 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.765456    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt" event={"ID":"509892d1-d707-4ce4-936a-b44c374c3e29","Type":"ContainerDied","Data":"189177b29180daba2eb513f17a00a4c2987ac9e0b4627efb7580d75712312d87"}
Jul 29 16:21:11.765651 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.765495    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt" event={"ID":"509892d1-d707-4ce4-936a-b44c374c3e29","Type":"ContainerDied","Data":"f1388c6a7d149dd674f739948d639c04d34156d9f66767dc2b3dad496bac8f13"}
Jul 29 16:21:11.765651 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.765512    2560 scope.go:117] "RemoveContainer" containerID="189177b29180daba2eb513f17a00a4c2987ac9e0b4627efb7580d75712312d87"
Jul 29 16:21:11.774938 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.774889    2560 scope.go:117] "RemoveContainer" containerID="189177b29180daba2eb513f17a00a4c2987ac9e0b4627efb7580d75712312d87"
Jul 29 16:21:11.775210 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:21:11.775193    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"189177b29180daba2eb513f17a00a4c2987ac9e0b4627efb7580d75712312d87\": container with ID starting with 189177b29180daba2eb513f17a00a4c2987ac9e0b4627efb7580d75712312d87 not found: ID does not exist" containerID="189177b29180daba2eb513f17a00a4c2987ac9e0b4627efb7580d75712312d87"
Jul 29 16:21:11.775265 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.775218    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"189177b29180daba2eb513f17a00a4c2987ac9e0b4627efb7580d75712312d87"} err="failed to get container status \"189177b29180daba2eb513f17a00a4c2987ac9e0b4627efb7580d75712312d87\": rpc error: code = NotFound desc = could not find container \"189177b29180daba2eb513f17a00a4c2987ac9e0b4627efb7580d75712312d87\": container with ID starting with 189177b29180daba2eb513f17a00a4c2987ac9e0b4627efb7580d75712312d87 not found: ID does not exist"
Jul 29 16:21:11.789314 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.789288    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"]
Jul 29 16:21:11.793245 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:11.793224    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-c6a1f06a/e2e-lifecycle-model-c6a1f06a-kserve-967fcccf5-v2rgt"]
Jul 29 16:21:12.043631 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:12.043536    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="509892d1-d707-4ce4-936a-b44c374c3e29" path="/var/lib/kubelet/pods/509892d1-d707-4ce4-936a-b44c374c3e29/volumes"
Jul 29 16:21:51.651532 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:21:51.651496    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:22:04.078421 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:04.078388    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:22:47.764882 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.764803    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"]
Jul 29 16:22:47.765481 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.765151    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e302cd3-a8a7-443e-8c6d-338dca091eb7" containerName="payload-pre-processing"
Jul 29 16:22:47.765481 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.765162    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e302cd3-a8a7-443e-8c6d-338dca091eb7" containerName="payload-pre-processing"
Jul 29 16:22:47.765481 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.765177    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="509892d1-d707-4ce4-936a-b44c374c3e29" containerName="main"
Jul 29 16:22:47.765481 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.765182    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="509892d1-d707-4ce4-936a-b44c374c3e29" containerName="main"
Jul 29 16:22:47.765481 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.765194    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b82c56c4-d080-42b4-b95e-d7e7449dc36c" containerName="payload-processing"
Jul 29 16:22:47.765481 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.765201    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b82c56c4-d080-42b4-b95e-d7e7449dc36c" containerName="payload-processing"
Jul 29 16:22:47.765481 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.765264    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e302cd3-a8a7-443e-8c6d-338dca091eb7" containerName="payload-pre-processing"
Jul 29 16:22:47.765481 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.765271    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="509892d1-d707-4ce4-936a-b44c374c3e29" containerName="main"
Jul 29 16:22:47.765481 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.765279    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b82c56c4-d080-42b4-b95e-d7e7449dc36c" containerName="payload-processing"
Jul 29 16:22:47.768240 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.768224    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"
Jul 29 16:22:47.770816 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.770796    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-b340d6-dockercfg-mjtnp\""
Jul 29 16:22:47.770887 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.770867    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-b340d6\""
Jul 29 16:22:47.780628 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.780606    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"]
Jul 29 16:22:47.823274 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.823244    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgqvd\" (UniqueName: \"kubernetes.io/projected/a0e93880-78ba-44de-9d10-97fbec98be2a-kube-api-access-rgqvd\") pod \"payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr\" (UID: \"a0e93880-78ba-44de-9d10-97fbec98be2a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"
Jul 29 16:22:47.823403 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.823288    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0e93880-78ba-44de-9d10-97fbec98be2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr\" (UID: \"a0e93880-78ba-44de-9d10-97fbec98be2a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"
Jul 29 16:22:47.924668 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.924630    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgqvd\" (UniqueName: \"kubernetes.io/projected/a0e93880-78ba-44de-9d10-97fbec98be2a-kube-api-access-rgqvd\") pod \"payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr\" (UID: \"a0e93880-78ba-44de-9d10-97fbec98be2a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"
Jul 29 16:22:47.924837 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.924687    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0e93880-78ba-44de-9d10-97fbec98be2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr\" (UID: \"a0e93880-78ba-44de-9d10-97fbec98be2a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"
Jul 29 16:22:47.925454 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.925434    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0e93880-78ba-44de-9d10-97fbec98be2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr\" (UID: \"a0e93880-78ba-44de-9d10-97fbec98be2a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"
Jul 29 16:22:47.932636 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:47.932610    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgqvd\" (UniqueName: \"kubernetes.io/projected/a0e93880-78ba-44de-9d10-97fbec98be2a-kube-api-access-rgqvd\") pod \"payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr\" (UID: \"a0e93880-78ba-44de-9d10-97fbec98be2a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"
Jul 29 16:22:48.077753 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:48.077667    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"
Jul 29 16:22:48.207023 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:48.206999    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"]
Jul 29 16:22:48.209511 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:22:48.209485    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e93880_78ba_44de_9d10_97fbec98be2a.slice/crio-c0f1d67471a8f4f5132c83ac56bb5c03eac5619833384c6ca1431c4851ac08be WatchSource:0}: Error finding container c0f1d67471a8f4f5132c83ac56bb5c03eac5619833384c6ca1431c4851ac08be: Status 404 returned error can't find the container with id c0f1d67471a8f4f5132c83ac56bb5c03eac5619833384c6ca1431c4851ac08be
Jul 29 16:22:49.210149 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:49.210115    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr" event={"ID":"a0e93880-78ba-44de-9d10-97fbec98be2a","Type":"ContainerStarted","Data":"8ad66f03eff93ac130944e0d77dab49e59a2dcad50cc40aa6e7e2605131ee5b3"}
Jul 29 16:22:49.210530 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:49.210162    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr" event={"ID":"a0e93880-78ba-44de-9d10-97fbec98be2a","Type":"ContainerStarted","Data":"c0f1d67471a8f4f5132c83ac56bb5c03eac5619833384c6ca1431c4851ac08be"}
Jul 29 16:22:49.210530 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:49.210265    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"
Jul 29 16:22:49.224910 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:49.224857    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr" podStartSLOduration=2.224843813 podStartE2EDuration="2.224843813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:22:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:22:49.224680474 +0000 UTC m=+2173.666886736" watchObservedRunningTime="2026-07-29 16:22:49.224843813 +0000 UTC m=+2173.667050072"
Jul 29 16:22:52.956595 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:22:52.956562    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:23:00.217350 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:00.217320    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"
Jul 29 16:23:15.815927 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.815879    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"]
Jul 29 16:23:15.819118 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.819102    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"
Jul 29 16:23:15.824394 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.824373    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-8d3dc6-dockercfg-86jfs\""
Jul 29 16:23:15.825118 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.825092    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-8d3dc6\""
Jul 29 16:23:15.831235 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.831213    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"]
Jul 29 16:23:15.866727 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.866684    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcqct\" (UniqueName: \"kubernetes.io/projected/40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4-kube-api-access-pcqct\") pod \"payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j\" (UID: \"40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4\") " pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"
Jul 29 16:23:15.866940 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.866748    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j\" (UID: \"40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4\") " pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"
Jul 29 16:23:15.868696 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.868674    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"]
Jul 29 16:23:15.871960 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.871945    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"
Jul 29 16:23:15.883777 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.883751    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"]
Jul 29 16:23:15.967491 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.967459    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx56q\" (UniqueName: \"kubernetes.io/projected/1ea4a690-182c-4aea-916b-05e5f4e798b2-kube-api-access-dx56q\") pod \"payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc\" (UID: \"1ea4a690-182c-4aea-916b-05e5f4e798b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"
Jul 29 16:23:15.967671 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.967501    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea4a690-182c-4aea-916b-05e5f4e798b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc\" (UID: \"1ea4a690-182c-4aea-916b-05e5f4e798b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"
Jul 29 16:23:15.967671 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.967571    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcqct\" (UniqueName: \"kubernetes.io/projected/40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4-kube-api-access-pcqct\") pod \"payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j\" (UID: \"40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4\") " pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"
Jul 29 16:23:15.967671 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.967618    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j\" (UID: \"40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4\") " pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"
Jul 29 16:23:15.968336 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.968320    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j\" (UID: \"40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4\") " pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"
Jul 29 16:23:15.975834 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:15.975812    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcqct\" (UniqueName: \"kubernetes.io/projected/40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4-kube-api-access-pcqct\") pod \"payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j\" (UID: \"40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4\") " pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"
Jul 29 16:23:16.068648 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:16.068567    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx56q\" (UniqueName: \"kubernetes.io/projected/1ea4a690-182c-4aea-916b-05e5f4e798b2-kube-api-access-dx56q\") pod \"payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc\" (UID: \"1ea4a690-182c-4aea-916b-05e5f4e798b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"
Jul 29 16:23:16.068648 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:16.068606    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea4a690-182c-4aea-916b-05e5f4e798b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc\" (UID: \"1ea4a690-182c-4aea-916b-05e5f4e798b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"
Jul 29 16:23:16.069368 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:16.069347    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea4a690-182c-4aea-916b-05e5f4e798b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc\" (UID: \"1ea4a690-182c-4aea-916b-05e5f4e798b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"
Jul 29 16:23:16.076575 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:16.076543    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx56q\" (UniqueName: \"kubernetes.io/projected/1ea4a690-182c-4aea-916b-05e5f4e798b2-kube-api-access-dx56q\") pod \"payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc\" (UID: \"1ea4a690-182c-4aea-916b-05e5f4e798b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"
Jul 29 16:23:16.130299 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:16.130264    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"
Jul 29 16:23:16.182377 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:16.182349    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"
Jul 29 16:23:16.273808 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:16.273772    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"]
Jul 29 16:23:16.274514 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:23:16.274485    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40daf23c_aaaa_4f6a_bf38_ce5900d2f5e4.slice/crio-cc67a002641ba972d10c337219435421920b3d05a7eea55a77f4de8fb47baff1 WatchSource:0}: Error finding container cc67a002641ba972d10c337219435421920b3d05a7eea55a77f4de8fb47baff1: Status 404 returned error can't find the container with id cc67a002641ba972d10c337219435421920b3d05a7eea55a77f4de8fb47baff1
Jul 29 16:23:16.330851 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:16.330777    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"]
Jul 29 16:23:16.332885 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:23:16.332857    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ea4a690_182c_4aea_916b_05e5f4e798b2.slice/crio-45d5c1de9e9f07b0951042c890bcb37efb0922b3cff822b4847f2f040defa8ab WatchSource:0}: Error finding container 45d5c1de9e9f07b0951042c890bcb37efb0922b3cff822b4847f2f040defa8ab: Status 404 returned error can't find the container with id 45d5c1de9e9f07b0951042c890bcb37efb0922b3cff822b4847f2f040defa8ab
Jul 29 16:23:16.341840 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:16.341811    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc" event={"ID":"1ea4a690-182c-4aea-916b-05e5f4e798b2","Type":"ContainerStarted","Data":"45d5c1de9e9f07b0951042c890bcb37efb0922b3cff822b4847f2f040defa8ab"}
Jul 29 16:23:16.343192 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:16.343172    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j" event={"ID":"40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4","Type":"ContainerStarted","Data":"cc67a002641ba972d10c337219435421920b3d05a7eea55a77f4de8fb47baff1"}
Jul 29 16:23:17.349283 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:17.349187    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc" event={"ID":"1ea4a690-182c-4aea-916b-05e5f4e798b2","Type":"ContainerStarted","Data":"6df5dbaf2e2b38ea074ed957584f7ce2e8e1ea3106d8b2e3b12e152e929e6426"}
Jul 29 16:23:17.349676 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:17.349281    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"
Jul 29 16:23:17.350760 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:17.350740    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j" event={"ID":"40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4","Type":"ContainerStarted","Data":"9e4182089c9066fc7be8e5eb56d8ab0c393d0da68b3b8069a720319d9c5f7e3d"}
Jul 29 16:23:17.350839 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:17.350809    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"
Jul 29 16:23:17.373044 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:17.373003    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc" podStartSLOduration=2.372990674 podStartE2EDuration="2.372990674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:23:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:23:17.371983505 +0000 UTC m=+2201.814189765" watchObservedRunningTime="2026-07-29 16:23:17.372990674 +0000 UTC m=+2201.815196911"
Jul 29 16:23:17.396627 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:17.396578    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j" podStartSLOduration=2.3965629 podStartE2EDuration="2.3965629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:23:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:23:17.39496955 +0000 UTC m=+2201.837175809" watchObservedRunningTime="2026-07-29 16:23:17.3965629 +0000 UTC m=+2201.838769162"
Jul 29 16:23:20.259240 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:20.259205    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:23:28.358286 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:28.358254    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"
Jul 29 16:23:28.358707 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:28.358326    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"
Jul 29 16:23:51.770694 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.770652    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"]
Jul 29 16:23:51.774225 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.774207    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.778346 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.778317    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b340d6\"/\"default-dockercfg-zdjt6\""
Jul 29 16:23:51.778346 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.778336    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b340d6\"/\"kube-root-ca.crt\""
Jul 29 16:23:51.778520 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.778328    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b340d6\"/\"openshift-service-ca.crt\""
Jul 29 16:23:51.778520 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.778349    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b340d6\"/\"test-model-b340d6-kserve-self-signed-certs\""
Jul 29 16:23:51.787290 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.787271    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"]
Jul 29 16:23:51.867816 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.867785    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-model-cache\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.867816 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.867816    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-dshm\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.868042 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.867883    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-tls-certs\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.868042 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.867971    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-home\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.868042 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.868000    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-tmp-dir\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.868165 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.868051    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28kn4\" (UniqueName: \"kubernetes.io/projected/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-kube-api-access-28kn4\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.969078 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.969048    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-home\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.969235 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.969081    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-tmp-dir\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.969235 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.969114    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28kn4\" (UniqueName: \"kubernetes.io/projected/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-kube-api-access-28kn4\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.969235 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.969151    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-model-cache\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.969235 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.969178    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-dshm\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.969465 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.969246    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-tls-certs\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.969546 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.969523    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-home\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.969603 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.969558    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-tmp-dir\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.969603 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.969579    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-model-cache\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.971460 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.971442    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-dshm\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.971705 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.971689    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-tls-certs\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:51.978324 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:51.978302    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28kn4\" (UniqueName: \"kubernetes.io/projected/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-kube-api-access-28kn4\") pod \"test-model-b340d6-kserve-66f4596b99-dtqkz\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") " pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:52.083662 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:52.083583    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:23:52.217874 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:52.217843    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"]
Jul 29 16:23:52.223183 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:23:52.223153    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod663e31fd_1f62_4dfa_9745_3b6b12bd3d29.slice/crio-4d1344fb9bdda3d6841a926c7311e59662093996428eb9b6df3aad1935cc8b37 WatchSource:0}: Error finding container 4d1344fb9bdda3d6841a926c7311e59662093996428eb9b6df3aad1935cc8b37: Status 404 returned error can't find the container with id 4d1344fb9bdda3d6841a926c7311e59662093996428eb9b6df3aad1935cc8b37
Jul 29 16:23:52.518278 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:52.518238    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz" event={"ID":"663e31fd-1f62-4dfa-9745-3b6b12bd3d29","Type":"ContainerStarted","Data":"fedce5d75852421ac8116ed14095f869b08f8214c95acc0b5fc411e9d800fbfd"}
Jul 29 16:23:52.518278 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:52.518280    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz" event={"ID":"663e31fd-1f62-4dfa-9745-3b6b12bd3d29","Type":"ContainerStarted","Data":"4d1344fb9bdda3d6841a926c7311e59662093996428eb9b6df3aad1935cc8b37"}
Jul 29 16:23:52.536007 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:52.535956    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz" podStartSLOduration=1.5359409689999999 podStartE2EDuration="1.535940969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:23:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:23:52.534409942 +0000 UTC m=+2236.976616214" watchObservedRunningTime="2026-07-29 16:23:52.535940969 +0000 UTC m=+2236.978147228"
Jul 29 16:23:55.057160 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:23:55.057131    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:24:02.083769 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:02.083679    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:24:02.083769 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:02.083731    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:24:02.096176 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:02.096150    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:24:02.581066 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:02.581034    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:24:06.654100 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:06.654069    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:24:16.152811 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.152773    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"]
Jul 29 16:24:16.156198 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.156174    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.160857 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.160834    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8d3dc6\"/\"test-model-8d3dc6-kserve-self-signed-certs\""
Jul 29 16:24:16.160857 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.160844    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8d3dc6\"/\"default-dockercfg-bcsfb\""
Jul 29 16:24:16.161055 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.160846    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8d3dc6\"/\"kube-root-ca.crt\""
Jul 29 16:24:16.161055 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.160878    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8d3dc6\"/\"openshift-service-ca.crt\""
Jul 29 16:24:16.168292 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.168269    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"]
Jul 29 16:24:16.179781 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.179761    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-tmp-dir\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.179905 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.179803    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c6baa45-43e6-4d4b-a609-0cb4209f045f-tls-certs\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.179974 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.179950    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8x4d\" (UniqueName: \"kubernetes.io/projected/6c6baa45-43e6-4d4b-a609-0cb4209f045f-kube-api-access-q8x4d\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.180024 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.179994    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-home\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.180067 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.180033    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-model-cache\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.180113 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.180093    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-dshm\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.280654 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.280611    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-home\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.280654 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.280658    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-model-cache\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.280906 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.280696    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-dshm\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.280906 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.280728    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-tmp-dir\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.280906 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.280752    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c6baa45-43e6-4d4b-a609-0cb4209f045f-tls-certs\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.280906 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.280783    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8x4d\" (UniqueName: \"kubernetes.io/projected/6c6baa45-43e6-4d4b-a609-0cb4209f045f-kube-api-access-q8x4d\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.281159 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.281108    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-home\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.281159 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.281144    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-model-cache\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.281233 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.281209    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-tmp-dir\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.283106 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.283083    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-dshm\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.283307 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.283289    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c6baa45-43e6-4d4b-a609-0cb4209f045f-tls-certs\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.289382 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.289359    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8x4d\" (UniqueName: \"kubernetes.io/projected/6c6baa45-43e6-4d4b-a609-0cb4209f045f-kube-api-access-q8x4d\") pod \"test-model-8d3dc6-kserve-7947c97957-vsnfv\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") " pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.465807 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.465731    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:16.599111 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.599087    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"]
Jul 29 16:24:16.601191 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:24:16.601165    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c6baa45_43e6_4d4b_a609_0cb4209f045f.slice/crio-c9bc4eba99e30f358761a6d46c93bcd7378b4697975002c2efbd02c21ee64ce0 WatchSource:0}: Error finding container c9bc4eba99e30f358761a6d46c93bcd7378b4697975002c2efbd02c21ee64ce0: Status 404 returned error can't find the container with id c9bc4eba99e30f358761a6d46c93bcd7378b4697975002c2efbd02c21ee64ce0
Jul 29 16:24:16.638785 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:16.638758    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv" event={"ID":"6c6baa45-43e6-4d4b-a609-0cb4209f045f","Type":"ContainerStarted","Data":"c9bc4eba99e30f358761a6d46c93bcd7378b4697975002c2efbd02c21ee64ce0"}
Jul 29 16:24:17.645381 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:17.645345    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv" event={"ID":"6c6baa45-43e6-4d4b-a609-0cb4209f045f","Type":"ContainerStarted","Data":"41ba58cbea4603c16ad8bd0510088261ed8cabfb5fc7de6a8b8ecdc1e2f971f3"}
Jul 29 16:24:17.674487 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:17.674439    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv" podStartSLOduration=1.674423649 podStartE2EDuration="1.674423649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:24:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:24:17.672866216 +0000 UTC m=+2262.115072474" watchObservedRunningTime="2026-07-29 16:24:17.674423649 +0000 UTC m=+2262.116629908"
Jul 29 16:24:22.065888 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:22.065853    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:24:26.466443 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:26.466404    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:26.467075 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:26.466463    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:26.479507 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:26.479482    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:26.700302 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:26.700273    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:24:59.483100 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:24:59.483066    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"]
Jul 29 16:25:00.625230 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:00.625196    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"]
Jul 29 16:25:04.762105 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:04.762061    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:25:11.287801 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.287766    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"]
Jul 29 16:25:11.288247 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.287988    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr" podUID="a0e93880-78ba-44de-9d10-97fbec98be2a" containerName="payload-pre-processing" containerID="cri-o://8ad66f03eff93ac130944e0d77dab49e59a2dcad50cc40aa6e7e2605131ee5b3" gracePeriod=30
Jul 29 16:25:11.538319 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.538261    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"
Jul 29 16:25:11.663893 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.663853    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0e93880-78ba-44de-9d10-97fbec98be2a-plugins-config-volume\") pod \"a0e93880-78ba-44de-9d10-97fbec98be2a\" (UID: \"a0e93880-78ba-44de-9d10-97fbec98be2a\") "
Jul 29 16:25:11.663893 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.663897    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgqvd\" (UniqueName: \"kubernetes.io/projected/a0e93880-78ba-44de-9d10-97fbec98be2a-kube-api-access-rgqvd\") pod \"a0e93880-78ba-44de-9d10-97fbec98be2a\" (UID: \"a0e93880-78ba-44de-9d10-97fbec98be2a\") "
Jul 29 16:25:11.664290 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.664262    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0e93880-78ba-44de-9d10-97fbec98be2a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a0e93880-78ba-44de-9d10-97fbec98be2a" (UID: "a0e93880-78ba-44de-9d10-97fbec98be2a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:25:11.666074 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.666056    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0e93880-78ba-44de-9d10-97fbec98be2a-kube-api-access-rgqvd" (OuterVolumeSpecName: "kube-api-access-rgqvd") pod "a0e93880-78ba-44de-9d10-97fbec98be2a" (UID: "a0e93880-78ba-44de-9d10-97fbec98be2a"). InnerVolumeSpecName "kube-api-access-rgqvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:25:11.765312 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.765279    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0e93880-78ba-44de-9d10-97fbec98be2a-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:11.765312 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.765306    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgqvd\" (UniqueName: \"kubernetes.io/projected/a0e93880-78ba-44de-9d10-97fbec98be2a-kube-api-access-rgqvd\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:11.907716 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.907679    2560 generic.go:356] "Generic (PLEG): container finished" podID="a0e93880-78ba-44de-9d10-97fbec98be2a" containerID="8ad66f03eff93ac130944e0d77dab49e59a2dcad50cc40aa6e7e2605131ee5b3" exitCode=0
Jul 29 16:25:11.907870 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.907752    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"
Jul 29 16:25:11.907870 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.907766    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr" event={"ID":"a0e93880-78ba-44de-9d10-97fbec98be2a","Type":"ContainerDied","Data":"8ad66f03eff93ac130944e0d77dab49e59a2dcad50cc40aa6e7e2605131ee5b3"}
Jul 29 16:25:11.907870 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.907856    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr" event={"ID":"a0e93880-78ba-44de-9d10-97fbec98be2a","Type":"ContainerDied","Data":"c0f1d67471a8f4f5132c83ac56bb5c03eac5619833384c6ca1431c4851ac08be"}
Jul 29 16:25:11.908008 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.907881    2560 scope.go:117] "RemoveContainer" containerID="8ad66f03eff93ac130944e0d77dab49e59a2dcad50cc40aa6e7e2605131ee5b3"
Jul 29 16:25:11.916875 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.916857    2560 scope.go:117] "RemoveContainer" containerID="8ad66f03eff93ac130944e0d77dab49e59a2dcad50cc40aa6e7e2605131ee5b3"
Jul 29 16:25:11.917156 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:25:11.917138    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ad66f03eff93ac130944e0d77dab49e59a2dcad50cc40aa6e7e2605131ee5b3\": container with ID starting with 8ad66f03eff93ac130944e0d77dab49e59a2dcad50cc40aa6e7e2605131ee5b3 not found: ID does not exist" containerID="8ad66f03eff93ac130944e0d77dab49e59a2dcad50cc40aa6e7e2605131ee5b3"
Jul 29 16:25:11.917210 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.917164    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ad66f03eff93ac130944e0d77dab49e59a2dcad50cc40aa6e7e2605131ee5b3"} err="failed to get container status \"8ad66f03eff93ac130944e0d77dab49e59a2dcad50cc40aa6e7e2605131ee5b3\": rpc error: code = NotFound desc = could not find container \"8ad66f03eff93ac130944e0d77dab49e59a2dcad50cc40aa6e7e2605131ee5b3\": container with ID starting with 8ad66f03eff93ac130944e0d77dab49e59a2dcad50cc40aa6e7e2605131ee5b3 not found: ID does not exist"
Jul 29 16:25:11.929846 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.929816    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"]
Jul 29 16:25:11.940451 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:11.940431    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b340d6-58fdc5f6f6-m62cr"]
Jul 29 16:25:12.043398 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:12.043366    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0e93880-78ba-44de-9d10-97fbec98be2a" path="/var/lib/kubelet/pods/a0e93880-78ba-44de-9d10-97fbec98be2a/volumes"
Jul 29 16:25:14.501696 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.501656    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz" podUID="663e31fd-1f62-4dfa-9745-3b6b12bd3d29" containerName="main" containerID="cri-o://fedce5d75852421ac8116ed14095f869b08f8214c95acc0b5fc411e9d800fbfd" gracePeriod=45
Jul 29 16:25:14.739125 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.739101    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:25:14.793314 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.793240    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-dshm\") pod \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") "
Jul 29 16:25:14.793314 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.793285    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28kn4\" (UniqueName: \"kubernetes.io/projected/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-kube-api-access-28kn4\") pod \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") "
Jul 29 16:25:14.793314 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.793305    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-model-cache\") pod \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") "
Jul 29 16:25:14.793609 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.793336    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-tls-certs\") pod \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") "
Jul 29 16:25:14.793609 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.793407    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-tmp-dir\") pod \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") "
Jul 29 16:25:14.793609 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.793450    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-home\") pod \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\" (UID: \"663e31fd-1f62-4dfa-9745-3b6b12bd3d29\") "
Jul 29 16:25:14.793797 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.793617    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-model-cache" (OuterVolumeSpecName: "model-cache") pod "663e31fd-1f62-4dfa-9745-3b6b12bd3d29" (UID: "663e31fd-1f62-4dfa-9745-3b6b12bd3d29"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:25:14.793797 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.793693    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "663e31fd-1f62-4dfa-9745-3b6b12bd3d29" (UID: "663e31fd-1f62-4dfa-9745-3b6b12bd3d29"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:25:14.793797 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.793754    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-model-cache\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:14.793797 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.793780    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-home" (OuterVolumeSpecName: "home") pod "663e31fd-1f62-4dfa-9745-3b6b12bd3d29" (UID: "663e31fd-1f62-4dfa-9745-3b6b12bd3d29"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:25:14.795562 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.795533    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "663e31fd-1f62-4dfa-9745-3b6b12bd3d29" (UID: "663e31fd-1f62-4dfa-9745-3b6b12bd3d29"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:25:14.795562 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.795543    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-dshm" (OuterVolumeSpecName: "dshm") pod "663e31fd-1f62-4dfa-9745-3b6b12bd3d29" (UID: "663e31fd-1f62-4dfa-9745-3b6b12bd3d29"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:25:14.795562 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.795551    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-kube-api-access-28kn4" (OuterVolumeSpecName: "kube-api-access-28kn4") pod "663e31fd-1f62-4dfa-9745-3b6b12bd3d29" (UID: "663e31fd-1f62-4dfa-9745-3b6b12bd3d29"). InnerVolumeSpecName "kube-api-access-28kn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:25:14.894816 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.894784    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-home\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:14.894816 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.894810    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-dshm\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:14.894816 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.894821    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28kn4\" (UniqueName: \"kubernetes.io/projected/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-kube-api-access-28kn4\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:14.895061 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.894831    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-tls-certs\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:14.895061 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.894839    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/663e31fd-1f62-4dfa-9745-3b6b12bd3d29-tmp-dir\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:14.924844 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.924817    2560 generic.go:356] "Generic (PLEG): container finished" podID="663e31fd-1f62-4dfa-9745-3b6b12bd3d29" containerID="fedce5d75852421ac8116ed14095f869b08f8214c95acc0b5fc411e9d800fbfd" exitCode=0
Jul 29 16:25:14.925006 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.924858    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"
Jul 29 16:25:14.925006 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.924907    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz" event={"ID":"663e31fd-1f62-4dfa-9745-3b6b12bd3d29","Type":"ContainerDied","Data":"fedce5d75852421ac8116ed14095f869b08f8214c95acc0b5fc411e9d800fbfd"}
Jul 29 16:25:14.925006 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.924976    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz" event={"ID":"663e31fd-1f62-4dfa-9745-3b6b12bd3d29","Type":"ContainerDied","Data":"4d1344fb9bdda3d6841a926c7311e59662093996428eb9b6df3aad1935cc8b37"}
Jul 29 16:25:14.925006 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.924999    2560 scope.go:117] "RemoveContainer" containerID="fedce5d75852421ac8116ed14095f869b08f8214c95acc0b5fc411e9d800fbfd"
Jul 29 16:25:14.934171 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.934154    2560 scope.go:117] "RemoveContainer" containerID="fedce5d75852421ac8116ed14095f869b08f8214c95acc0b5fc411e9d800fbfd"
Jul 29 16:25:14.934417 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:25:14.934401    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fedce5d75852421ac8116ed14095f869b08f8214c95acc0b5fc411e9d800fbfd\": container with ID starting with fedce5d75852421ac8116ed14095f869b08f8214c95acc0b5fc411e9d800fbfd not found: ID does not exist" containerID="fedce5d75852421ac8116ed14095f869b08f8214c95acc0b5fc411e9d800fbfd"
Jul 29 16:25:14.934482 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.934424    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fedce5d75852421ac8116ed14095f869b08f8214c95acc0b5fc411e9d800fbfd"} err="failed to get container status \"fedce5d75852421ac8116ed14095f869b08f8214c95acc0b5fc411e9d800fbfd\": rpc error: code = NotFound desc = could not find container \"fedce5d75852421ac8116ed14095f869b08f8214c95acc0b5fc411e9d800fbfd\": container with ID starting with fedce5d75852421ac8116ed14095f869b08f8214c95acc0b5fc411e9d800fbfd not found: ID does not exist"
Jul 29 16:25:14.946867 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.946842    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"]
Jul 29 16:25:14.949897 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:14.949878    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-b340d6/test-model-b340d6-kserve-66f4596b99-dtqkz"]
Jul 29 16:25:15.644254 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:15.644207    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv" podUID="6c6baa45-43e6-4d4b-a609-0cb4209f045f" containerName="main" containerID="cri-o://41ba58cbea4603c16ad8bd0510088261ed8cabfb5fc7de6a8b8ecdc1e2f971f3" gracePeriod=45
Jul 29 16:25:15.888987 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:15.888959    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:25:15.932524 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:15.932441    2560 generic.go:356] "Generic (PLEG): container finished" podID="6c6baa45-43e6-4d4b-a609-0cb4209f045f" containerID="41ba58cbea4603c16ad8bd0510088261ed8cabfb5fc7de6a8b8ecdc1e2f971f3" exitCode=0
Jul 29 16:25:15.932524 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:15.932506    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv" event={"ID":"6c6baa45-43e6-4d4b-a609-0cb4209f045f","Type":"ContainerDied","Data":"41ba58cbea4603c16ad8bd0510088261ed8cabfb5fc7de6a8b8ecdc1e2f971f3"}
Jul 29 16:25:15.932524 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:15.932511    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"
Jul 29 16:25:15.932759 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:15.932536    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv" event={"ID":"6c6baa45-43e6-4d4b-a609-0cb4209f045f","Type":"ContainerDied","Data":"c9bc4eba99e30f358761a6d46c93bcd7378b4697975002c2efbd02c21ee64ce0"}
Jul 29 16:25:15.932759 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:15.932553    2560 scope.go:117] "RemoveContainer" containerID="41ba58cbea4603c16ad8bd0510088261ed8cabfb5fc7de6a8b8ecdc1e2f971f3"
Jul 29 16:25:15.941131 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:15.941112    2560 scope.go:117] "RemoveContainer" containerID="41ba58cbea4603c16ad8bd0510088261ed8cabfb5fc7de6a8b8ecdc1e2f971f3"
Jul 29 16:25:15.941379 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:25:15.941361    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41ba58cbea4603c16ad8bd0510088261ed8cabfb5fc7de6a8b8ecdc1e2f971f3\": container with ID starting with 41ba58cbea4603c16ad8bd0510088261ed8cabfb5fc7de6a8b8ecdc1e2f971f3 not found: ID does not exist" containerID="41ba58cbea4603c16ad8bd0510088261ed8cabfb5fc7de6a8b8ecdc1e2f971f3"
Jul 29 16:25:15.941436 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:15.941388    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41ba58cbea4603c16ad8bd0510088261ed8cabfb5fc7de6a8b8ecdc1e2f971f3"} err="failed to get container status \"41ba58cbea4603c16ad8bd0510088261ed8cabfb5fc7de6a8b8ecdc1e2f971f3\": rpc error: code = NotFound desc = could not find container \"41ba58cbea4603c16ad8bd0510088261ed8cabfb5fc7de6a8b8ecdc1e2f971f3\": container with ID starting with 41ba58cbea4603c16ad8bd0510088261ed8cabfb5fc7de6a8b8ecdc1e2f971f3 not found: ID does not exist"
Jul 29 16:25:16.003684 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.003654    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-home\") pod \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") "
Jul 29 16:25:16.003684 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.003684    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-dshm\") pod \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") "
Jul 29 16:25:16.003899 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.003737    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c6baa45-43e6-4d4b-a609-0cb4209f045f-tls-certs\") pod \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") "
Jul 29 16:25:16.003899 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.003755    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-model-cache\") pod \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") "
Jul 29 16:25:16.003899 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.003783    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8x4d\" (UniqueName: \"kubernetes.io/projected/6c6baa45-43e6-4d4b-a609-0cb4209f045f-kube-api-access-q8x4d\") pod \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") "
Jul 29 16:25:16.003899 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.003806    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-tmp-dir\") pod \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\" (UID: \"6c6baa45-43e6-4d4b-a609-0cb4209f045f\") "
Jul 29 16:25:16.004142 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.003927    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-home" (OuterVolumeSpecName: "home") pod "6c6baa45-43e6-4d4b-a609-0cb4209f045f" (UID: "6c6baa45-43e6-4d4b-a609-0cb4209f045f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:25:16.004142 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.004056    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-model-cache" (OuterVolumeSpecName: "model-cache") pod "6c6baa45-43e6-4d4b-a609-0cb4209f045f" (UID: "6c6baa45-43e6-4d4b-a609-0cb4209f045f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:25:16.004240 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.004131    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6c6baa45-43e6-4d4b-a609-0cb4209f045f" (UID: "6c6baa45-43e6-4d4b-a609-0cb4209f045f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:25:16.004240 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.004157    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-model-cache\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:16.004240 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.004175    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-home\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:16.006120 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.006092    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-dshm" (OuterVolumeSpecName: "dshm") pod "6c6baa45-43e6-4d4b-a609-0cb4209f045f" (UID: "6c6baa45-43e6-4d4b-a609-0cb4209f045f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:25:16.006230 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.006115    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c6baa45-43e6-4d4b-a609-0cb4209f045f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6c6baa45-43e6-4d4b-a609-0cb4209f045f" (UID: "6c6baa45-43e6-4d4b-a609-0cb4209f045f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:25:16.006230 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.006125    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c6baa45-43e6-4d4b-a609-0cb4209f045f-kube-api-access-q8x4d" (OuterVolumeSpecName: "kube-api-access-q8x4d") pod "6c6baa45-43e6-4d4b-a609-0cb4209f045f" (UID: "6c6baa45-43e6-4d4b-a609-0cb4209f045f"). InnerVolumeSpecName "kube-api-access-q8x4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:25:16.044550 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.044521    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="663e31fd-1f62-4dfa-9745-3b6b12bd3d29" path="/var/lib/kubelet/pods/663e31fd-1f62-4dfa-9745-3b6b12bd3d29/volumes"
Jul 29 16:25:16.105515 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.105484    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-dshm\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:16.105515 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.105510    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c6baa45-43e6-4d4b-a609-0cb4209f045f-tls-certs\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:16.105674 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.105524    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8x4d\" (UniqueName: \"kubernetes.io/projected/6c6baa45-43e6-4d4b-a609-0cb4209f045f-kube-api-access-q8x4d\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:16.105674 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.105535    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c6baa45-43e6-4d4b-a609-0cb4209f045f-tmp-dir\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:16.265711 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.265627    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"]
Jul 29 16:25:16.273806 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:16.273777    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-8d3dc6/test-model-8d3dc6-kserve-7947c97957-vsnfv"]
Jul 29 16:25:18.042968 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:18.042898    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c6baa45-43e6-4d4b-a609-0cb4209f045f" path="/var/lib/kubelet/pods/6c6baa45-43e6-4d4b-a609-0cb4209f045f/volumes"
Jul 29 16:25:21.172288 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:21.172247    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:25:24.068506 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.068471    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"]
Jul 29 16:25:24.068981 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.068713    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j" podUID="40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4" containerName="payload-processing" containerID="cri-o://9e4182089c9066fc7be8e5eb56d8ab0c393d0da68b3b8069a720319d9c5f7e3d" gracePeriod=30
Jul 29 16:25:24.109100 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.109059    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"]
Jul 29 16:25:24.109399 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.109345    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc" podUID="1ea4a690-182c-4aea-916b-05e5f4e798b2" containerName="payload-pre-processing" containerID="cri-o://6df5dbaf2e2b38ea074ed957584f7ce2e8e1ea3106d8b2e3b12e152e929e6426" gracePeriod=30
Jul 29 16:25:24.353527 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.353503    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"
Jul 29 16:25:24.382295 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.382272    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"
Jul 29 16:25:24.475691 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.475656    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4-plugins-config-volume\") pod \"40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4\" (UID: \"40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4\") "
Jul 29 16:25:24.475888 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.475707    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcqct\" (UniqueName: \"kubernetes.io/projected/40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4-kube-api-access-pcqct\") pod \"40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4\" (UID: \"40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4\") "
Jul 29 16:25:24.475888 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.475745    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea4a690-182c-4aea-916b-05e5f4e798b2-plugins-config-volume\") pod \"1ea4a690-182c-4aea-916b-05e5f4e798b2\" (UID: \"1ea4a690-182c-4aea-916b-05e5f4e798b2\") "
Jul 29 16:25:24.475888 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.475858    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dx56q\" (UniqueName: \"kubernetes.io/projected/1ea4a690-182c-4aea-916b-05e5f4e798b2-kube-api-access-dx56q\") pod \"1ea4a690-182c-4aea-916b-05e5f4e798b2\" (UID: \"1ea4a690-182c-4aea-916b-05e5f4e798b2\") "
Jul 29 16:25:24.476183 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.476155    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4" (UID: "40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:25:24.476256 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.476188    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ea4a690-182c-4aea-916b-05e5f4e798b2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1ea4a690-182c-4aea-916b-05e5f4e798b2" (UID: "1ea4a690-182c-4aea-916b-05e5f4e798b2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:25:24.477779 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.477754    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4-kube-api-access-pcqct" (OuterVolumeSpecName: "kube-api-access-pcqct") pod "40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4" (UID: "40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4"). InnerVolumeSpecName "kube-api-access-pcqct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:25:24.477879 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.477776    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ea4a690-182c-4aea-916b-05e5f4e798b2-kube-api-access-dx56q" (OuterVolumeSpecName: "kube-api-access-dx56q") pod "1ea4a690-182c-4aea-916b-05e5f4e798b2" (UID: "1ea4a690-182c-4aea-916b-05e5f4e798b2"). InnerVolumeSpecName "kube-api-access-dx56q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:25:24.577470 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.577436    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcqct\" (UniqueName: \"kubernetes.io/projected/40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4-kube-api-access-pcqct\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:24.577470 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.577463    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea4a690-182c-4aea-916b-05e5f4e798b2-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:24.577470 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.577473    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx56q\" (UniqueName: \"kubernetes.io/projected/1ea4a690-182c-4aea-916b-05e5f4e798b2-kube-api-access-dx56q\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:24.577470 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.577482    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:25:24.979755 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.979723    2560 generic.go:356] "Generic (PLEG): container finished" podID="1ea4a690-182c-4aea-916b-05e5f4e798b2" containerID="6df5dbaf2e2b38ea074ed957584f7ce2e8e1ea3106d8b2e3b12e152e929e6426" exitCode=0
Jul 29 16:25:24.979958 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.979765    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"
Jul 29 16:25:24.979958 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.979800    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc" event={"ID":"1ea4a690-182c-4aea-916b-05e5f4e798b2","Type":"ContainerDied","Data":"6df5dbaf2e2b38ea074ed957584f7ce2e8e1ea3106d8b2e3b12e152e929e6426"}
Jul 29 16:25:24.979958 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.979843    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc" event={"ID":"1ea4a690-182c-4aea-916b-05e5f4e798b2","Type":"ContainerDied","Data":"45d5c1de9e9f07b0951042c890bcb37efb0922b3cff822b4847f2f040defa8ab"}
Jul 29 16:25:24.979958 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.979863    2560 scope.go:117] "RemoveContainer" containerID="6df5dbaf2e2b38ea074ed957584f7ce2e8e1ea3106d8b2e3b12e152e929e6426"
Jul 29 16:25:24.981094 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.981072    2560 generic.go:356] "Generic (PLEG): container finished" podID="40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4" containerID="9e4182089c9066fc7be8e5eb56d8ab0c393d0da68b3b8069a720319d9c5f7e3d" exitCode=0
Jul 29 16:25:24.981238 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.981112    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"
Jul 29 16:25:24.981238 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.981153    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j" event={"ID":"40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4","Type":"ContainerDied","Data":"9e4182089c9066fc7be8e5eb56d8ab0c393d0da68b3b8069a720319d9c5f7e3d"}
Jul 29 16:25:24.981238 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.981198    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j" event={"ID":"40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4","Type":"ContainerDied","Data":"cc67a002641ba972d10c337219435421920b3d05a7eea55a77f4de8fb47baff1"}
Jul 29 16:25:24.988282 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.988262    2560 scope.go:117] "RemoveContainer" containerID="6df5dbaf2e2b38ea074ed957584f7ce2e8e1ea3106d8b2e3b12e152e929e6426"
Jul 29 16:25:24.988545 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:25:24.988520    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6df5dbaf2e2b38ea074ed957584f7ce2e8e1ea3106d8b2e3b12e152e929e6426\": container with ID starting with 6df5dbaf2e2b38ea074ed957584f7ce2e8e1ea3106d8b2e3b12e152e929e6426 not found: ID does not exist" containerID="6df5dbaf2e2b38ea074ed957584f7ce2e8e1ea3106d8b2e3b12e152e929e6426"
Jul 29 16:25:24.988676 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.988554    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6df5dbaf2e2b38ea074ed957584f7ce2e8e1ea3106d8b2e3b12e152e929e6426"} err="failed to get container status \"6df5dbaf2e2b38ea074ed957584f7ce2e8e1ea3106d8b2e3b12e152e929e6426\": rpc error: code = NotFound desc = could not find container \"6df5dbaf2e2b38ea074ed957584f7ce2e8e1ea3106d8b2e3b12e152e929e6426\": container with ID starting with 6df5dbaf2e2b38ea074ed957584f7ce2e8e1ea3106d8b2e3b12e152e929e6426 not found: ID does not exist"
Jul 29 16:25:24.988676 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.988577    2560 scope.go:117] "RemoveContainer" containerID="9e4182089c9066fc7be8e5eb56d8ab0c393d0da68b3b8069a720319d9c5f7e3d"
Jul 29 16:25:24.996363 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.996347    2560 scope.go:117] "RemoveContainer" containerID="9e4182089c9066fc7be8e5eb56d8ab0c393d0da68b3b8069a720319d9c5f7e3d"
Jul 29 16:25:24.996577 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:25:24.996558    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e4182089c9066fc7be8e5eb56d8ab0c393d0da68b3b8069a720319d9c5f7e3d\": container with ID starting with 9e4182089c9066fc7be8e5eb56d8ab0c393d0da68b3b8069a720319d9c5f7e3d not found: ID does not exist" containerID="9e4182089c9066fc7be8e5eb56d8ab0c393d0da68b3b8069a720319d9c5f7e3d"
Jul 29 16:25:24.996627 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:24.996585    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e4182089c9066fc7be8e5eb56d8ab0c393d0da68b3b8069a720319d9c5f7e3d"} err="failed to get container status \"9e4182089c9066fc7be8e5eb56d8ab0c393d0da68b3b8069a720319d9c5f7e3d\": rpc error: code = NotFound desc = could not find container \"9e4182089c9066fc7be8e5eb56d8ab0c393d0da68b3b8069a720319d9c5f7e3d\": container with ID starting with 9e4182089c9066fc7be8e5eb56d8ab0c393d0da68b3b8069a720319d9c5f7e3d not found: ID does not exist"
Jul 29 16:25:25.013862 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:25.013838    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"]
Jul 29 16:25:25.019991 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:25.019967    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-8d3dc6-75d798478d-72t5j"]
Jul 29 16:25:25.037510 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:25.037489    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"]
Jul 29 16:25:25.051346 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:25.051325    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8d3dc6-674668fc9b-v57nc"]
Jul 29 16:25:26.042859 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:26.042821    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ea4a690-182c-4aea-916b-05e5f4e798b2" path="/var/lib/kubelet/pods/1ea4a690-182c-4aea-916b-05e5f4e798b2/volumes"
Jul 29 16:25:26.043249 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:26.043219    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4" path="/var/lib/kubelet/pods/40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4/volumes"
Jul 29 16:25:31.576778 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:31.576697    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:25:37.013968 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.013935    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"]
Jul 29 16:25:37.014378 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014290    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="663e31fd-1f62-4dfa-9745-3b6b12bd3d29" containerName="main"
Jul 29 16:25:37.014378 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014302    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="663e31fd-1f62-4dfa-9745-3b6b12bd3d29" containerName="main"
Jul 29 16:25:37.014378 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014318    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ea4a690-182c-4aea-916b-05e5f4e798b2" containerName="payload-pre-processing"
Jul 29 16:25:37.014378 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014325    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ea4a690-182c-4aea-916b-05e5f4e798b2" containerName="payload-pre-processing"
Jul 29 16:25:37.014378 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014336    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c6baa45-43e6-4d4b-a609-0cb4209f045f" containerName="main"
Jul 29 16:25:37.014378 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014344    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c6baa45-43e6-4d4b-a609-0cb4209f045f" containerName="main"
Jul 29 16:25:37.014378 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014353    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e93880-78ba-44de-9d10-97fbec98be2a" containerName="payload-pre-processing"
Jul 29 16:25:37.014378 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014358    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e93880-78ba-44de-9d10-97fbec98be2a" containerName="payload-pre-processing"
Jul 29 16:25:37.014378 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014364    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4" containerName="payload-processing"
Jul 29 16:25:37.014378 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014369    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4" containerName="payload-processing"
Jul 29 16:25:37.014760 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014431    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e93880-78ba-44de-9d10-97fbec98be2a" containerName="payload-pre-processing"
Jul 29 16:25:37.014760 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014441    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ea4a690-182c-4aea-916b-05e5f4e798b2" containerName="payload-pre-processing"
Jul 29 16:25:37.014760 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014447    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="663e31fd-1f62-4dfa-9745-3b6b12bd3d29" containerName="main"
Jul 29 16:25:37.014760 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014456    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c6baa45-43e6-4d4b-a609-0cb4209f045f" containerName="main"
Jul 29 16:25:37.014760 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.014461    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="40daf23c-aaaa-4f6a-bf38-ce5900d2f5e4" containerName="payload-processing"
Jul 29 16:25:37.019078 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.019058    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"
Jul 29 16:25:37.022576 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.022553    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ef8cba\""
Jul 29 16:25:37.022670 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.022598    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ef8cba-dockercfg-5427g\""
Jul 29 16:25:37.028038 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.028013    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"]
Jul 29 16:25:37.182826 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.182796    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxqxj\" (UniqueName: \"kubernetes.io/projected/6a3ac0da-486a-485c-969d-d0a7d0cce41f-kube-api-access-mxqxj\") pod \"payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb\" (UID: \"6a3ac0da-486a-485c-969d-d0a7d0cce41f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"
Jul 29 16:25:37.182995 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.182889    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a3ac0da-486a-485c-969d-d0a7d0cce41f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb\" (UID: \"6a3ac0da-486a-485c-969d-d0a7d0cce41f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"
Jul 29 16:25:37.229225 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.229195    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"]
Jul 29 16:25:37.232691 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.232673    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"
Jul 29 16:25:37.258532 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.258498    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"]
Jul 29 16:25:37.283355 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.283292    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a3ac0da-486a-485c-969d-d0a7d0cce41f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb\" (UID: \"6a3ac0da-486a-485c-969d-d0a7d0cce41f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"
Jul 29 16:25:37.283355 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.283334    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxqxj\" (UniqueName: \"kubernetes.io/projected/6a3ac0da-486a-485c-969d-d0a7d0cce41f-kube-api-access-mxqxj\") pod \"payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb\" (UID: \"6a3ac0da-486a-485c-969d-d0a7d0cce41f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"
Jul 29 16:25:37.283950 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.283930    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a3ac0da-486a-485c-969d-d0a7d0cce41f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb\" (UID: \"6a3ac0da-486a-485c-969d-d0a7d0cce41f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"
Jul 29 16:25:37.294130 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.294108    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxqxj\" (UniqueName: \"kubernetes.io/projected/6a3ac0da-486a-485c-969d-d0a7d0cce41f-kube-api-access-mxqxj\") pod \"payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb\" (UID: \"6a3ac0da-486a-485c-969d-d0a7d0cce41f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"
Jul 29 16:25:37.329572 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.329549    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"
Jul 29 16:25:37.384030 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.383991    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bpxn\" (UniqueName: \"kubernetes.io/projected/04ccb886-cd04-48af-aade-b91f87d17b33-kube-api-access-9bpxn\") pod \"payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth\" (UID: \"04ccb886-cd04-48af-aade-b91f87d17b33\") " pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"
Jul 29 16:25:37.384187 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.384087    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04ccb886-cd04-48af-aade-b91f87d17b33-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth\" (UID: \"04ccb886-cd04-48af-aade-b91f87d17b33\") " pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"
Jul 29 16:25:37.471511 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.471415    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"]
Jul 29 16:25:37.473812 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:25:37.473778    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a3ac0da_486a_485c_969d_d0a7d0cce41f.slice/crio-3601d69b075f00dc7e8d1d463c15ea5549a2522c8db72ab03381a4eeee04989c WatchSource:0}: Error finding container 3601d69b075f00dc7e8d1d463c15ea5549a2522c8db72ab03381a4eeee04989c: Status 404 returned error can't find the container with id 3601d69b075f00dc7e8d1d463c15ea5549a2522c8db72ab03381a4eeee04989c
Jul 29 16:25:37.485199 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.485173    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bpxn\" (UniqueName: \"kubernetes.io/projected/04ccb886-cd04-48af-aade-b91f87d17b33-kube-api-access-9bpxn\") pod \"payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth\" (UID: \"04ccb886-cd04-48af-aade-b91f87d17b33\") " pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"
Jul 29 16:25:37.485298 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.485240    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04ccb886-cd04-48af-aade-b91f87d17b33-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth\" (UID: \"04ccb886-cd04-48af-aade-b91f87d17b33\") " pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"
Jul 29 16:25:37.485878 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.485850    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04ccb886-cd04-48af-aade-b91f87d17b33-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth\" (UID: \"04ccb886-cd04-48af-aade-b91f87d17b33\") " pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"
Jul 29 16:25:37.510171 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.510147    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bpxn\" (UniqueName: \"kubernetes.io/projected/04ccb886-cd04-48af-aade-b91f87d17b33-kube-api-access-9bpxn\") pod \"payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth\" (UID: \"04ccb886-cd04-48af-aade-b91f87d17b33\") " pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"
Jul 29 16:25:37.542643 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.542587    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"
Jul 29 16:25:37.690749 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:37.690721    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"]
Jul 29 16:25:37.692823 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:25:37.692798    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04ccb886_cd04_48af_aade_b91f87d17b33.slice/crio-8dd13b090e6bc54cdab5a3fcd973d394fa9f8e3551f7566079afe05367ff4aae WatchSource:0}: Error finding container 8dd13b090e6bc54cdab5a3fcd973d394fa9f8e3551f7566079afe05367ff4aae: Status 404 returned error can't find the container with id 8dd13b090e6bc54cdab5a3fcd973d394fa9f8e3551f7566079afe05367ff4aae
Jul 29 16:25:38.046745 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:38.046714    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb" event={"ID":"6a3ac0da-486a-485c-969d-d0a7d0cce41f","Type":"ContainerStarted","Data":"4af0bf9538aa73c1b116e4f95d9c24aaed8646b3d8dd460cceffc96ad60a67f9"}
Jul 29 16:25:38.047169 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:38.046764    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb" event={"ID":"6a3ac0da-486a-485c-969d-d0a7d0cce41f","Type":"ContainerStarted","Data":"3601d69b075f00dc7e8d1d463c15ea5549a2522c8db72ab03381a4eeee04989c"}
Jul 29 16:25:38.047169 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:38.046789    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"
Jul 29 16:25:38.048212 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:38.048190    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth" event={"ID":"04ccb886-cd04-48af-aade-b91f87d17b33","Type":"ContainerStarted","Data":"c433a9c693ba2b99b77e346530a06686e2087d09196791b8c3ad26f65439e370"}
Jul 29 16:25:38.048282 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:38.048230    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth" event={"ID":"04ccb886-cd04-48af-aade-b91f87d17b33","Type":"ContainerStarted","Data":"8dd13b090e6bc54cdab5a3fcd973d394fa9f8e3551f7566079afe05367ff4aae"}
Jul 29 16:25:38.048282 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:38.048249    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"
Jul 29 16:25:38.070601 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:38.070558    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb" podStartSLOduration=2.070543754 podStartE2EDuration="2.070543754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:25:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:25:38.069184475 +0000 UTC m=+2342.511390918" watchObservedRunningTime="2026-07-29 16:25:38.070543754 +0000 UTC m=+2342.512750012"
Jul 29 16:25:38.094594 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:38.094553    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth" podStartSLOduration=1.094541533 podStartE2EDuration="1.094541533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:25:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:25:38.093528749 +0000 UTC m=+2342.535735007" watchObservedRunningTime="2026-07-29 16:25:38.094541533 +0000 UTC m=+2342.536747792"
Jul 29 16:25:43.094510 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:43.094482    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:25:49.055262 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:49.055236    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"
Jul 29 16:25:49.055735 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:25:49.055301    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"
Jul 29 16:26:06.395357 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:06.395323    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"]
Jul 29 16:26:06.398863 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:06.398845    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"
Jul 29 16:26:06.406723 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:06.406700    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-6d4da6-dockercfg-psj6z\""
Jul 29 16:26:06.406847 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:06.406769    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-6d4da6\""
Jul 29 16:26:06.417972 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:06.417951    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82a361fc-93bb-4a2c-ab03-4da1ac4e67d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt\" (UID: \"82a361fc-93bb-4a2c-ab03-4da1ac4e67d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"
Jul 29 16:26:06.418050 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:06.417988    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5fdn\" (UniqueName: \"kubernetes.io/projected/82a361fc-93bb-4a2c-ab03-4da1ac4e67d1-kube-api-access-b5fdn\") pod \"payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt\" (UID: \"82a361fc-93bb-4a2c-ab03-4da1ac4e67d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"
Jul 29 16:26:06.513665 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:06.513636    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"]
Jul 29 16:26:06.518422 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:06.518396    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82a361fc-93bb-4a2c-ab03-4da1ac4e67d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt\" (UID: \"82a361fc-93bb-4a2c-ab03-4da1ac4e67d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"
Jul 29 16:26:06.518560 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:06.518431    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5fdn\" (UniqueName: \"kubernetes.io/projected/82a361fc-93bb-4a2c-ab03-4da1ac4e67d1-kube-api-access-b5fdn\") pod \"payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt\" (UID: \"82a361fc-93bb-4a2c-ab03-4da1ac4e67d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"
Jul 29 16:26:06.519144 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:06.519126    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82a361fc-93bb-4a2c-ab03-4da1ac4e67d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt\" (UID: \"82a361fc-93bb-4a2c-ab03-4da1ac4e67d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"
Jul 29 16:26:06.541414 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:06.541385    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5fdn\" (UniqueName: \"kubernetes.io/projected/82a361fc-93bb-4a2c-ab03-4da1ac4e67d1-kube-api-access-b5fdn\") pod \"payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt\" (UID: \"82a361fc-93bb-4a2c-ab03-4da1ac4e67d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"
Jul 29 16:26:06.709894 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:06.709806    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"
Jul 29 16:26:06.876886 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:06.876857    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"]
Jul 29 16:26:06.878208 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:26:06.878177    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82a361fc_93bb_4a2c_ab03_4da1ac4e67d1.slice/crio-0939be2ad329e073c45f3af88b1c38d76afebfbcf66cba7ee83acb2ae11f50bb WatchSource:0}: Error finding container 0939be2ad329e073c45f3af88b1c38d76afebfbcf66cba7ee83acb2ae11f50bb: Status 404 returned error can't find the container with id 0939be2ad329e073c45f3af88b1c38d76afebfbcf66cba7ee83acb2ae11f50bb
Jul 29 16:26:07.202266 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:07.202223    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt" event={"ID":"82a361fc-93bb-4a2c-ab03-4da1ac4e67d1","Type":"ContainerStarted","Data":"39202cceac40283aa7cad88ca396e34f6079e525c8f6e0f3cb59d89f68e9db3a"}
Jul 29 16:26:07.202266 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:07.202273    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt" event={"ID":"82a361fc-93bb-4a2c-ab03-4da1ac4e67d1","Type":"ContainerStarted","Data":"0939be2ad329e073c45f3af88b1c38d76afebfbcf66cba7ee83acb2ae11f50bb"}
Jul 29 16:26:07.202575 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:07.202357    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"
Jul 29 16:26:07.241425 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:07.241375    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt" podStartSLOduration=1.241357507 podStartE2EDuration="1.241357507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:26:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:26:07.240133001 +0000 UTC m=+2371.682339260" watchObservedRunningTime="2026-07-29 16:26:07.241357507 +0000 UTC m=+2371.683563799"
Jul 29 16:26:13.003792 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:13.003751    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:26:18.210087 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:18.210058    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"
Jul 29 16:26:32.886466 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:32.886432    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh"]
Jul 29 16:26:32.889948 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:32.889911    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh"
Jul 29 16:26:32.892641 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:32.892614    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n57pr\""
Jul 29 16:26:32.892641 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:32.892644    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 16:26:32.898060 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:32.898033    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh"]
Jul 29 16:26:33.045031 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:33.044993    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eccd2baa-252c-4170-bfa2-91e005c42852-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh\" (UID: \"eccd2baa-252c-4170-bfa2-91e005c42852\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh"
Jul 29 16:26:33.145620 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:33.145586    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eccd2baa-252c-4170-bfa2-91e005c42852-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh\" (UID: \"eccd2baa-252c-4170-bfa2-91e005c42852\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh"
Jul 29 16:26:33.146300 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:33.146275    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eccd2baa-252c-4170-bfa2-91e005c42852-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh\" (UID: \"eccd2baa-252c-4170-bfa2-91e005c42852\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh"
Jul 29 16:26:33.201176 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:33.201150    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh"
Jul 29 16:26:33.337543 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:33.337519    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh"]
Jul 29 16:26:33.339602 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:26:33.339574    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeccd2baa_252c_4170_bfa2_91e005c42852.slice/crio-1b23514852c11602cf9c0c22566b0aa3c345ec6e7dd484d6c9e431b8c5390d05 WatchSource:0}: Error finding container 1b23514852c11602cf9c0c22566b0aa3c345ec6e7dd484d6c9e431b8c5390d05: Status 404 returned error can't find the container with id 1b23514852c11602cf9c0c22566b0aa3c345ec6e7dd484d6c9e431b8c5390d05
Jul 29 16:26:33.341472 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:33.341450    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 16:26:34.337042 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:34.337008    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh" event={"ID":"eccd2baa-252c-4170-bfa2-91e005c42852","Type":"ContainerStarted","Data":"1b23514852c11602cf9c0c22566b0aa3c345ec6e7dd484d6c9e431b8c5390d05"}
Jul 29 16:26:36.347886 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:36.347780    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh" event={"ID":"eccd2baa-252c-4170-bfa2-91e005c42852","Type":"ContainerStarted","Data":"e0d99d297bce66d509ec191bb5ebfbbffe63b775a647f5b0ca8feb83eca582c1"}
Jul 29 16:26:36.369965 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:36.369906    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh" podStartSLOduration=1.919774416 podStartE2EDuration="4.369892321s" totalImagesPullingTime="2.450117905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:26:36.36747624 +0000 UTC m=+2400.809682495" watchObservedRunningTime="2026-07-29 16:26:36.369892321 +0000 UTC m=+2400.812098593"
Jul 29 16:26:37.354543 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:37.354509    2560 generic.go:356] "Generic (PLEG): container finished" podID="eccd2baa-252c-4170-bfa2-91e005c42852" containerID="e0d99d297bce66d509ec191bb5ebfbbffe63b775a647f5b0ca8feb83eca582c1" exitCode=0
Jul 29 16:26:37.354979 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:37.354597    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh" event={"ID":"eccd2baa-252c-4170-bfa2-91e005c42852","Type":"ContainerDied","Data":"e0d99d297bce66d509ec191bb5ebfbbffe63b775a647f5b0ca8feb83eca582c1"}
Jul 29 16:26:38.505937 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:38.505892    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh"
Jul 29 16:26:38.598144 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:38.598113    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eccd2baa-252c-4170-bfa2-91e005c42852-maas-api-service-ca\") pod \"eccd2baa-252c-4170-bfa2-91e005c42852\" (UID: \"eccd2baa-252c-4170-bfa2-91e005c42852\") "
Jul 29 16:26:38.598497 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:38.598479    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eccd2baa-252c-4170-bfa2-91e005c42852-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eccd2baa-252c-4170-bfa2-91e005c42852" (UID: "eccd2baa-252c-4170-bfa2-91e005c42852"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:26:38.699784 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:38.699751    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eccd2baa-252c-4170-bfa2-91e005c42852-maas-api-service-ca\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:26:39.364880 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:39.364850    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh"
Jul 29 16:26:39.365091 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:39.364890    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh" event={"ID":"eccd2baa-252c-4170-bfa2-91e005c42852","Type":"ContainerDied","Data":"1b23514852c11602cf9c0c22566b0aa3c345ec6e7dd484d6c9e431b8c5390d05"}
Jul 29 16:26:39.365091 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:39.364951    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b23514852c11602cf9c0c22566b0aa3c345ec6e7dd484d6c9e431b8c5390d05"
Jul 29 16:26:48.709322 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:48.709286    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"]
Jul 29 16:26:48.709732 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:48.709578    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth" podUID="04ccb886-cd04-48af-aade-b91f87d17b33" containerName="payload-processing" containerID="cri-o://c433a9c693ba2b99b77e346530a06686e2087d09196791b8c3ad26f65439e370" gracePeriod=30
Jul 29 16:26:48.744136 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:48.744108    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"]
Jul 29 16:26:48.744343 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:48.744303    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb" podUID="6a3ac0da-486a-485c-969d-d0a7d0cce41f" containerName="payload-pre-processing" containerID="cri-o://4af0bf9538aa73c1b116e4f95d9c24aaed8646b3d8dd460cceffc96ad60a67f9" gracePeriod=30
Jul 29 16:26:48.872463 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:48.872431    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh"]
Jul 29 16:26:48.877017 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:48.876991    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ef8cba-n6jdh"]
Jul 29 16:26:49.010312 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.010286    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"
Jul 29 16:26:49.013039 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.013022    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"
Jul 29 16:26:49.093687 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.093655    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bpxn\" (UniqueName: \"kubernetes.io/projected/04ccb886-cd04-48af-aade-b91f87d17b33-kube-api-access-9bpxn\") pod \"04ccb886-cd04-48af-aade-b91f87d17b33\" (UID: \"04ccb886-cd04-48af-aade-b91f87d17b33\") "
Jul 29 16:26:49.093867 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.093702    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxqxj\" (UniqueName: \"kubernetes.io/projected/6a3ac0da-486a-485c-969d-d0a7d0cce41f-kube-api-access-mxqxj\") pod \"6a3ac0da-486a-485c-969d-d0a7d0cce41f\" (UID: \"6a3ac0da-486a-485c-969d-d0a7d0cce41f\") "
Jul 29 16:26:49.093867 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.093752    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a3ac0da-486a-485c-969d-d0a7d0cce41f-plugins-config-volume\") pod \"6a3ac0da-486a-485c-969d-d0a7d0cce41f\" (UID: \"6a3ac0da-486a-485c-969d-d0a7d0cce41f\") "
Jul 29 16:26:49.093996 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.093878    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04ccb886-cd04-48af-aade-b91f87d17b33-plugins-config-volume\") pod \"04ccb886-cd04-48af-aade-b91f87d17b33\" (UID: \"04ccb886-cd04-48af-aade-b91f87d17b33\") "
Jul 29 16:26:49.094301 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.094254    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a3ac0da-486a-485c-969d-d0a7d0cce41f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6a3ac0da-486a-485c-969d-d0a7d0cce41f" (UID: "6a3ac0da-486a-485c-969d-d0a7d0cce41f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:26:49.094301 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.094281    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04ccb886-cd04-48af-aade-b91f87d17b33-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "04ccb886-cd04-48af-aade-b91f87d17b33" (UID: "04ccb886-cd04-48af-aade-b91f87d17b33"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:26:49.095815 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.095794    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a3ac0da-486a-485c-969d-d0a7d0cce41f-kube-api-access-mxqxj" (OuterVolumeSpecName: "kube-api-access-mxqxj") pod "6a3ac0da-486a-485c-969d-d0a7d0cce41f" (UID: "6a3ac0da-486a-485c-969d-d0a7d0cce41f"). InnerVolumeSpecName "kube-api-access-mxqxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:26:49.095980 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.095820    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04ccb886-cd04-48af-aade-b91f87d17b33-kube-api-access-9bpxn" (OuterVolumeSpecName: "kube-api-access-9bpxn") pod "04ccb886-cd04-48af-aade-b91f87d17b33" (UID: "04ccb886-cd04-48af-aade-b91f87d17b33"). InnerVolumeSpecName "kube-api-access-9bpxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:26:49.195417 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.195389    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bpxn\" (UniqueName: \"kubernetes.io/projected/04ccb886-cd04-48af-aade-b91f87d17b33-kube-api-access-9bpxn\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:26:49.195417 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.195413    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxqxj\" (UniqueName: \"kubernetes.io/projected/6a3ac0da-486a-485c-969d-d0a7d0cce41f-kube-api-access-mxqxj\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:26:49.195417 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.195423    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a3ac0da-486a-485c-969d-d0a7d0cce41f-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:26:49.195608 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.195432    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04ccb886-cd04-48af-aade-b91f87d17b33-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:26:49.413602 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.413570    2560 generic.go:356] "Generic (PLEG): container finished" podID="6a3ac0da-486a-485c-969d-d0a7d0cce41f" containerID="4af0bf9538aa73c1b116e4f95d9c24aaed8646b3d8dd460cceffc96ad60a67f9" exitCode=0
Jul 29 16:26:49.413773 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.413617    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"
Jul 29 16:26:49.413773 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.413650    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb" event={"ID":"6a3ac0da-486a-485c-969d-d0a7d0cce41f","Type":"ContainerDied","Data":"4af0bf9538aa73c1b116e4f95d9c24aaed8646b3d8dd460cceffc96ad60a67f9"}
Jul 29 16:26:49.413773 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.413699    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb" event={"ID":"6a3ac0da-486a-485c-969d-d0a7d0cce41f","Type":"ContainerDied","Data":"3601d69b075f00dc7e8d1d463c15ea5549a2522c8db72ab03381a4eeee04989c"}
Jul 29 16:26:49.413773 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.413729    2560 scope.go:117] "RemoveContainer" containerID="4af0bf9538aa73c1b116e4f95d9c24aaed8646b3d8dd460cceffc96ad60a67f9"
Jul 29 16:26:49.415002 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.414980    2560 generic.go:356] "Generic (PLEG): container finished" podID="04ccb886-cd04-48af-aade-b91f87d17b33" containerID="c433a9c693ba2b99b77e346530a06686e2087d09196791b8c3ad26f65439e370" exitCode=0
Jul 29 16:26:49.415119 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.415008    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth" event={"ID":"04ccb886-cd04-48af-aade-b91f87d17b33","Type":"ContainerDied","Data":"c433a9c693ba2b99b77e346530a06686e2087d09196791b8c3ad26f65439e370"}
Jul 29 16:26:49.415119 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.415026    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"
Jul 29 16:26:49.415119 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.415055    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth" event={"ID":"04ccb886-cd04-48af-aade-b91f87d17b33","Type":"ContainerDied","Data":"8dd13b090e6bc54cdab5a3fcd973d394fa9f8e3551f7566079afe05367ff4aae"}
Jul 29 16:26:49.424130 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.424115    2560 scope.go:117] "RemoveContainer" containerID="4af0bf9538aa73c1b116e4f95d9c24aaed8646b3d8dd460cceffc96ad60a67f9"
Jul 29 16:26:49.424378 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:26:49.424356    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4af0bf9538aa73c1b116e4f95d9c24aaed8646b3d8dd460cceffc96ad60a67f9\": container with ID starting with 4af0bf9538aa73c1b116e4f95d9c24aaed8646b3d8dd460cceffc96ad60a67f9 not found: ID does not exist" containerID="4af0bf9538aa73c1b116e4f95d9c24aaed8646b3d8dd460cceffc96ad60a67f9"
Jul 29 16:26:49.424457 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.424384    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4af0bf9538aa73c1b116e4f95d9c24aaed8646b3d8dd460cceffc96ad60a67f9"} err="failed to get container status \"4af0bf9538aa73c1b116e4f95d9c24aaed8646b3d8dd460cceffc96ad60a67f9\": rpc error: code = NotFound desc = could not find container \"4af0bf9538aa73c1b116e4f95d9c24aaed8646b3d8dd460cceffc96ad60a67f9\": container with ID starting with 4af0bf9538aa73c1b116e4f95d9c24aaed8646b3d8dd460cceffc96ad60a67f9 not found: ID does not exist"
Jul 29 16:26:49.424457 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.424400    2560 scope.go:117] "RemoveContainer" containerID="c433a9c693ba2b99b77e346530a06686e2087d09196791b8c3ad26f65439e370"
Jul 29 16:26:49.431595 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.431580    2560 scope.go:117] "RemoveContainer" containerID="c433a9c693ba2b99b77e346530a06686e2087d09196791b8c3ad26f65439e370"
Jul 29 16:26:49.431838 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:26:49.431822    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c433a9c693ba2b99b77e346530a06686e2087d09196791b8c3ad26f65439e370\": container with ID starting with c433a9c693ba2b99b77e346530a06686e2087d09196791b8c3ad26f65439e370 not found: ID does not exist" containerID="c433a9c693ba2b99b77e346530a06686e2087d09196791b8c3ad26f65439e370"
Jul 29 16:26:49.431877 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.431846    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c433a9c693ba2b99b77e346530a06686e2087d09196791b8c3ad26f65439e370"} err="failed to get container status \"c433a9c693ba2b99b77e346530a06686e2087d09196791b8c3ad26f65439e370\": rpc error: code = NotFound desc = could not find container \"c433a9c693ba2b99b77e346530a06686e2087d09196791b8c3ad26f65439e370\": container with ID starting with c433a9c693ba2b99b77e346530a06686e2087d09196791b8c3ad26f65439e370 not found: ID does not exist"
Jul 29 16:26:49.436253 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.436233    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"]
Jul 29 16:26:49.442523 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.442503    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ef8cba-945879cb4-4tcgb"]
Jul 29 16:26:49.452271 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.452251    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"]
Jul 29 16:26:49.456239 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:49.456212    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ef8cba-77c54df86-6qpth"]
Jul 29 16:26:50.042804 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:50.042730    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04ccb886-cd04-48af-aade-b91f87d17b33" path="/var/lib/kubelet/pods/04ccb886-cd04-48af-aade-b91f87d17b33/volumes"
Jul 29 16:26:50.043176 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:50.043083    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a3ac0da-486a-485c-969d-d0a7d0cce41f" path="/var/lib/kubelet/pods/6a3ac0da-486a-485c-969d-d0a7d0cce41f/volumes"
Jul 29 16:26:50.043366 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:50.043353    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eccd2baa-252c-4170-bfa2-91e005c42852" path="/var/lib/kubelet/pods/eccd2baa-252c-4170-bfa2-91e005c42852/volumes"
Jul 29 16:26:51.564196 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:26:51.564158    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:27:07.035876 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.035793    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"]
Jul 29 16:27:07.036367 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.036064    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt" podUID="82a361fc-93bb-4a2c-ab03-4da1ac4e67d1" containerName="payload-pre-processing" containerID="cri-o://39202cceac40283aa7cad88ca396e34f6079e525c8f6e0f3cb59d89f68e9db3a" gracePeriod=30
Jul 29 16:27:07.291840 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.291773    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"
Jul 29 16:27:07.347329 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.347300    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82a361fc-93bb-4a2c-ab03-4da1ac4e67d1-plugins-config-volume\") pod \"82a361fc-93bb-4a2c-ab03-4da1ac4e67d1\" (UID: \"82a361fc-93bb-4a2c-ab03-4da1ac4e67d1\") "
Jul 29 16:27:07.347507 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.347343    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5fdn\" (UniqueName: \"kubernetes.io/projected/82a361fc-93bb-4a2c-ab03-4da1ac4e67d1-kube-api-access-b5fdn\") pod \"82a361fc-93bb-4a2c-ab03-4da1ac4e67d1\" (UID: \"82a361fc-93bb-4a2c-ab03-4da1ac4e67d1\") "
Jul 29 16:27:07.347653 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.347629    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82a361fc-93bb-4a2c-ab03-4da1ac4e67d1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "82a361fc-93bb-4a2c-ab03-4da1ac4e67d1" (UID: "82a361fc-93bb-4a2c-ab03-4da1ac4e67d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:27:07.349288 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.349256    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82a361fc-93bb-4a2c-ab03-4da1ac4e67d1-kube-api-access-b5fdn" (OuterVolumeSpecName: "kube-api-access-b5fdn") pod "82a361fc-93bb-4a2c-ab03-4da1ac4e67d1" (UID: "82a361fc-93bb-4a2c-ab03-4da1ac4e67d1"). InnerVolumeSpecName "kube-api-access-b5fdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:27:07.448336 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.448312    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82a361fc-93bb-4a2c-ab03-4da1ac4e67d1-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:27:07.448336 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.448334    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5fdn\" (UniqueName: \"kubernetes.io/projected/82a361fc-93bb-4a2c-ab03-4da1ac4e67d1-kube-api-access-b5fdn\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:27:07.505387 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.505355    2560 generic.go:356] "Generic (PLEG): container finished" podID="82a361fc-93bb-4a2c-ab03-4da1ac4e67d1" containerID="39202cceac40283aa7cad88ca396e34f6079e525c8f6e0f3cb59d89f68e9db3a" exitCode=0
Jul 29 16:27:07.505494 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.505401    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"
Jul 29 16:27:07.505494 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.505425    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt" event={"ID":"82a361fc-93bb-4a2c-ab03-4da1ac4e67d1","Type":"ContainerDied","Data":"39202cceac40283aa7cad88ca396e34f6079e525c8f6e0f3cb59d89f68e9db3a"}
Jul 29 16:27:07.505494 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.505474    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt" event={"ID":"82a361fc-93bb-4a2c-ab03-4da1ac4e67d1","Type":"ContainerDied","Data":"0939be2ad329e073c45f3af88b1c38d76afebfbcf66cba7ee83acb2ae11f50bb"}
Jul 29 16:27:07.505602 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.505503    2560 scope.go:117] "RemoveContainer" containerID="39202cceac40283aa7cad88ca396e34f6079e525c8f6e0f3cb59d89f68e9db3a"
Jul 29 16:27:07.514828 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.514810    2560 scope.go:117] "RemoveContainer" containerID="39202cceac40283aa7cad88ca396e34f6079e525c8f6e0f3cb59d89f68e9db3a"
Jul 29 16:27:07.515107 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:27:07.515086    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39202cceac40283aa7cad88ca396e34f6079e525c8f6e0f3cb59d89f68e9db3a\": container with ID starting with 39202cceac40283aa7cad88ca396e34f6079e525c8f6e0f3cb59d89f68e9db3a not found: ID does not exist" containerID="39202cceac40283aa7cad88ca396e34f6079e525c8f6e0f3cb59d89f68e9db3a"
Jul 29 16:27:07.515170 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.515116    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39202cceac40283aa7cad88ca396e34f6079e525c8f6e0f3cb59d89f68e9db3a"} err="failed to get container status \"39202cceac40283aa7cad88ca396e34f6079e525c8f6e0f3cb59d89f68e9db3a\": rpc error: code = NotFound desc = could not find container \"39202cceac40283aa7cad88ca396e34f6079e525c8f6e0f3cb59d89f68e9db3a\": container with ID starting with 39202cceac40283aa7cad88ca396e34f6079e525c8f6e0f3cb59d89f68e9db3a not found: ID does not exist"
Jul 29 16:27:07.528396 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.528374    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"]
Jul 29 16:27:07.532329 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:07.532308    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-6d4da6-7b956dcc56-b9xgt"]
Jul 29 16:27:08.042403 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:08.042372    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82a361fc-93bb-4a2c-ab03-4da1ac4e67d1" path="/var/lib/kubelet/pods/82a361fc-93bb-4a2c-ab03-4da1ac4e67d1/volumes"
Jul 29 16:27:09.773323 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:09.773291    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:27:19.262801 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.262762    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"]
Jul 29 16:27:19.263338 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.263290    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04ccb886-cd04-48af-aade-b91f87d17b33" containerName="payload-processing"
Jul 29 16:27:19.263338 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.263311    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="04ccb886-cd04-48af-aade-b91f87d17b33" containerName="payload-processing"
Jul 29 16:27:19.263338 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.263328    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a3ac0da-486a-485c-969d-d0a7d0cce41f" containerName="payload-pre-processing"
Jul 29 16:27:19.263338 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.263337    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a3ac0da-486a-485c-969d-d0a7d0cce41f" containerName="payload-pre-processing"
Jul 29 16:27:19.263557 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.263357    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82a361fc-93bb-4a2c-ab03-4da1ac4e67d1" containerName="payload-pre-processing"
Jul 29 16:27:19.263557 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.263366    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="82a361fc-93bb-4a2c-ab03-4da1ac4e67d1" containerName="payload-pre-processing"
Jul 29 16:27:19.263557 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.263380    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eccd2baa-252c-4170-bfa2-91e005c42852" containerName="revoke-keys"
Jul 29 16:27:19.263557 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.263390    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="eccd2baa-252c-4170-bfa2-91e005c42852" containerName="revoke-keys"
Jul 29 16:27:19.263557 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.263491    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a3ac0da-486a-485c-969d-d0a7d0cce41f" containerName="payload-pre-processing"
Jul 29 16:27:19.263557 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.263502    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="82a361fc-93bb-4a2c-ab03-4da1ac4e67d1" containerName="payload-pre-processing"
Jul 29 16:27:19.263557 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.263515    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="04ccb886-cd04-48af-aade-b91f87d17b33" containerName="payload-processing"
Jul 29 16:27:19.263557 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.263526    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="eccd2baa-252c-4170-bfa2-91e005c42852" containerName="revoke-keys"
Jul 29 16:27:19.266515 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.266495    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"
Jul 29 16:27:19.270429 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.270405    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-81ab8b\""
Jul 29 16:27:19.273798 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.273775    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-81ab8b-dockercfg-v766w\""
Jul 29 16:27:19.289133 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.289111    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"]
Jul 29 16:27:19.319232 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.319202    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"]
Jul 29 16:27:19.322620 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.322602    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"
Jul 29 16:27:19.335292 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.335270    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"]
Jul 29 16:27:19.354296 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.354276    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86b4617f-47ab-4ccc-aced-b2892a8b956e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq\" (UID: \"86b4617f-47ab-4ccc-aced-b2892a8b956e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"
Jul 29 16:27:19.354386 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.354339    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sg6m\" (UniqueName: \"kubernetes.io/projected/86b4617f-47ab-4ccc-aced-b2892a8b956e-kube-api-access-9sg6m\") pod \"payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq\" (UID: \"86b4617f-47ab-4ccc-aced-b2892a8b956e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"
Jul 29 16:27:19.455491 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.455457    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn6m4\" (UniqueName: \"kubernetes.io/projected/580c020b-a6cd-4185-92f2-f3184defdca2-kube-api-access-qn6m4\") pod \"payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb\" (UID: \"580c020b-a6cd-4185-92f2-f3184defdca2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"
Jul 29 16:27:19.455663 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.455498    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sg6m\" (UniqueName: \"kubernetes.io/projected/86b4617f-47ab-4ccc-aced-b2892a8b956e-kube-api-access-9sg6m\") pod \"payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq\" (UID: \"86b4617f-47ab-4ccc-aced-b2892a8b956e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"
Jul 29 16:27:19.455663 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.455584    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86b4617f-47ab-4ccc-aced-b2892a8b956e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq\" (UID: \"86b4617f-47ab-4ccc-aced-b2892a8b956e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"
Jul 29 16:27:19.455663 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.455616    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/580c020b-a6cd-4185-92f2-f3184defdca2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb\" (UID: \"580c020b-a6cd-4185-92f2-f3184defdca2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"
Jul 29 16:27:19.456355 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.456335    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86b4617f-47ab-4ccc-aced-b2892a8b956e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq\" (UID: \"86b4617f-47ab-4ccc-aced-b2892a8b956e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"
Jul 29 16:27:19.465278 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.465255    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sg6m\" (UniqueName: \"kubernetes.io/projected/86b4617f-47ab-4ccc-aced-b2892a8b956e-kube-api-access-9sg6m\") pod \"payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq\" (UID: \"86b4617f-47ab-4ccc-aced-b2892a8b956e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"
Jul 29 16:27:19.556701 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.556607    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/580c020b-a6cd-4185-92f2-f3184defdca2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb\" (UID: \"580c020b-a6cd-4185-92f2-f3184defdca2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"
Jul 29 16:27:19.556701 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.556682    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn6m4\" (UniqueName: \"kubernetes.io/projected/580c020b-a6cd-4185-92f2-f3184defdca2-kube-api-access-qn6m4\") pod \"payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb\" (UID: \"580c020b-a6cd-4185-92f2-f3184defdca2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"
Jul 29 16:27:19.557301 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.557278    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/580c020b-a6cd-4185-92f2-f3184defdca2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb\" (UID: \"580c020b-a6cd-4185-92f2-f3184defdca2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"
Jul 29 16:27:19.570314 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.570290    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn6m4\" (UniqueName: \"kubernetes.io/projected/580c020b-a6cd-4185-92f2-f3184defdca2-kube-api-access-qn6m4\") pod \"payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb\" (UID: \"580c020b-a6cd-4185-92f2-f3184defdca2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"
Jul 29 16:27:19.576107 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.576088    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"
Jul 29 16:27:19.632301 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.632271    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"
Jul 29 16:27:19.727200 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.727170    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"]
Jul 29 16:27:19.731425 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:27:19.731385    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86b4617f_47ab_4ccc_aced_b2892a8b956e.slice/crio-c92167b7ce9ca1c4d03da2434d91edbb69be4645e222045753282b66d40cdf85 WatchSource:0}: Error finding container c92167b7ce9ca1c4d03da2434d91edbb69be4645e222045753282b66d40cdf85: Status 404 returned error can't find the container with id c92167b7ce9ca1c4d03da2434d91edbb69be4645e222045753282b66d40cdf85
Jul 29 16:27:19.784355 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:19.784335    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"]
Jul 29 16:27:19.787954 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:27:19.787911    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod580c020b_a6cd_4185_92f2_f3184defdca2.slice/crio-4fa0fb23932bca72cbec11a1040c41dbee24b40c600be7a296ce4deb4c07c618 WatchSource:0}: Error finding container 4fa0fb23932bca72cbec11a1040c41dbee24b40c600be7a296ce4deb4c07c618: Status 404 returned error can't find the container with id 4fa0fb23932bca72cbec11a1040c41dbee24b40c600be7a296ce4deb4c07c618
Jul 29 16:27:20.568284 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:20.568246    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb" event={"ID":"580c020b-a6cd-4185-92f2-f3184defdca2","Type":"ContainerStarted","Data":"eec05a244b29a1cf6853fa8bec819e81b5a480813b7c4eca66fd36fb292a4720"}
Jul 29 16:27:20.568694 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:20.568397    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"
Jul 29 16:27:20.568694 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:20.568426    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb" event={"ID":"580c020b-a6cd-4185-92f2-f3184defdca2","Type":"ContainerStarted","Data":"4fa0fb23932bca72cbec11a1040c41dbee24b40c600be7a296ce4deb4c07c618"}
Jul 29 16:27:20.569702 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:20.569676    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq" event={"ID":"86b4617f-47ab-4ccc-aced-b2892a8b956e","Type":"ContainerStarted","Data":"df63bd664e8615ba34ccd1ce27a0bbb24c1f5c956e5bab159a6306ff05e1b207"}
Jul 29 16:27:20.569849 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:20.569727    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq" event={"ID":"86b4617f-47ab-4ccc-aced-b2892a8b956e","Type":"ContainerStarted","Data":"c92167b7ce9ca1c4d03da2434d91edbb69be4645e222045753282b66d40cdf85"}
Jul 29 16:27:20.569849 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:20.569757    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"
Jul 29 16:27:20.586326 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:20.586279    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb" podStartSLOduration=1.586265735 podStartE2EDuration="1.586265735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:27:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:27:20.585485867 +0000 UTC m=+2445.027692126" watchObservedRunningTime="2026-07-29 16:27:20.586265735 +0000 UTC m=+2445.028471971"
Jul 29 16:27:20.603595 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:20.603551    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq" podStartSLOduration=1.603541537 podStartE2EDuration="1.603541537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:27:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:27:20.601648931 +0000 UTC m=+2445.043855195" watchObservedRunningTime="2026-07-29 16:27:20.603541537 +0000 UTC m=+2445.045747795"
Jul 29 16:27:23.056885 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:23.056848    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:27:31.579283 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:31.579255    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"
Jul 29 16:27:31.579826 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:31.579807    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"
Jul 29 16:27:48.521111 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.521075    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"]
Jul 29 16:27:48.524483 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.524464    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"
Jul 29 16:27:48.527339 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.527308    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-eccd34\""
Jul 29 16:27:48.527473 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.527380    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-eccd34-dockercfg-j5svx\""
Jul 29 16:27:48.533828 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.533801    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"]
Jul 29 16:27:48.553780 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.553750    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"]
Jul 29 16:27:48.557140 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.557121    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"
Jul 29 16:27:48.566838 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.566812    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"]
Jul 29 16:27:48.612872 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.612828    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24m79\" (UniqueName: \"kubernetes.io/projected/ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab-kube-api-access-24m79\") pod \"payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v\" (UID: \"ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab\") " pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"
Jul 29 16:27:48.613074 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.612879    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npd9x\" (UniqueName: \"kubernetes.io/projected/b227d330-6988-4cb3-a5da-233000216a14-kube-api-access-npd9x\") pod \"payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr\" (UID: \"b227d330-6988-4cb3-a5da-233000216a14\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"
Jul 29 16:27:48.613074 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.612956    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b227d330-6988-4cb3-a5da-233000216a14-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr\" (UID: \"b227d330-6988-4cb3-a5da-233000216a14\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"
Jul 29 16:27:48.613074 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.613021    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v\" (UID: \"ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab\") " pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"
Jul 29 16:27:48.714436 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.714402    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v\" (UID: \"ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab\") " pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"
Jul 29 16:27:48.714628 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.714482    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24m79\" (UniqueName: \"kubernetes.io/projected/ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab-kube-api-access-24m79\") pod \"payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v\" (UID: \"ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab\") " pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"
Jul 29 16:27:48.714628 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.714510    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npd9x\" (UniqueName: \"kubernetes.io/projected/b227d330-6988-4cb3-a5da-233000216a14-kube-api-access-npd9x\") pod \"payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr\" (UID: \"b227d330-6988-4cb3-a5da-233000216a14\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"
Jul 29 16:27:48.714628 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.714535    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b227d330-6988-4cb3-a5da-233000216a14-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr\" (UID: \"b227d330-6988-4cb3-a5da-233000216a14\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"
Jul 29 16:27:48.715151 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.715128    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v\" (UID: \"ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab\") " pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"
Jul 29 16:27:48.715282 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.715264    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b227d330-6988-4cb3-a5da-233000216a14-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr\" (UID: \"b227d330-6988-4cb3-a5da-233000216a14\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"
Jul 29 16:27:48.729871 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.729837    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24m79\" (UniqueName: \"kubernetes.io/projected/ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab-kube-api-access-24m79\") pod \"payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v\" (UID: \"ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab\") " pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"
Jul 29 16:27:48.730904 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.730884    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npd9x\" (UniqueName: \"kubernetes.io/projected/b227d330-6988-4cb3-a5da-233000216a14-kube-api-access-npd9x\") pod \"payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr\" (UID: \"b227d330-6988-4cb3-a5da-233000216a14\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"
Jul 29 16:27:48.834501 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.834422    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"
Jul 29 16:27:48.867740 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.867711    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"
Jul 29 16:27:48.986656 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:48.986624    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"]
Jul 29 16:27:48.990069 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:27:48.990038    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecbfc11e_3ca6_4e76_bb86_cb35a9db83ab.slice/crio-6396adcda47df8bea68092255fbe8ff14f08bad722106bdd029edef2a52d86df WatchSource:0}: Error finding container 6396adcda47df8bea68092255fbe8ff14f08bad722106bdd029edef2a52d86df: Status 404 returned error can't find the container with id 6396adcda47df8bea68092255fbe8ff14f08bad722106bdd029edef2a52d86df
Jul 29 16:27:49.219544 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:49.219513    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"]
Jul 29 16:27:49.220347 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:27:49.220318    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb227d330_6988_4cb3_a5da_233000216a14.slice/crio-f4b672cc4b517ee35c3ba1c8ebd29aa372782687e27b706ff105a6753fc0497f WatchSource:0}: Error finding container f4b672cc4b517ee35c3ba1c8ebd29aa372782687e27b706ff105a6753fc0497f: Status 404 returned error can't find the container with id f4b672cc4b517ee35c3ba1c8ebd29aa372782687e27b706ff105a6753fc0497f
Jul 29 16:27:49.708503 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:49.708462    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v" event={"ID":"ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab","Type":"ContainerStarted","Data":"a5cb10ed1f1d84f1b0d9f8f1042ab9ed6a6abbcdbdcdfd969aa3dd7426a0858b"}
Jul 29 16:27:49.708996 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:49.708524    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v" event={"ID":"ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab","Type":"ContainerStarted","Data":"6396adcda47df8bea68092255fbe8ff14f08bad722106bdd029edef2a52d86df"}
Jul 29 16:27:49.708996 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:49.708594    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"
Jul 29 16:27:49.710006 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:49.709983    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr" event={"ID":"b227d330-6988-4cb3-a5da-233000216a14","Type":"ContainerStarted","Data":"83e6523a0ee5183bf44f5b6efdadd593272d6a8dda9f99167af3f2821345356d"}
Jul 29 16:27:49.710145 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:49.710019    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr" event={"ID":"b227d330-6988-4cb3-a5da-233000216a14","Type":"ContainerStarted","Data":"f4b672cc4b517ee35c3ba1c8ebd29aa372782687e27b706ff105a6753fc0497f"}
Jul 29 16:27:49.710145 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:49.710124    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"
Jul 29 16:27:49.730613 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:49.730566    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v" podStartSLOduration=1.7305530930000002 podStartE2EDuration="1.730553093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:27:49.728942028 +0000 UTC m=+2474.171148287" watchObservedRunningTime="2026-07-29 16:27:49.730553093 +0000 UTC m=+2474.172759386"
Jul 29 16:27:49.745310 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:49.745272    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr" podStartSLOduration=1.74525963 podStartE2EDuration="1.74525963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:27:49.743990496 +0000 UTC m=+2474.186196756" watchObservedRunningTime="2026-07-29 16:27:49.74525963 +0000 UTC m=+2474.187465888"
Jul 29 16:27:55.460933 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:27:55.460888    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:28:00.716512 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:00.716485    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"
Jul 29 16:28:00.716905 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:00.716551    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"
Jul 29 16:28:23.260898 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.260864    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"]
Jul 29 16:28:23.264585 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.264567    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.272398 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.272365    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-81ab8b\"/\"default-dockercfg-knk2z\""
Jul 29 16:28:23.272561 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.272434    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-81ab8b\"/\"kube-root-ca.crt\""
Jul 29 16:28:23.272561 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.272458    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-81ab8b\"/\"auth-test-model-81ab8b-kserve-self-signed-certs\""
Jul 29 16:28:23.272561 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.272506    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-81ab8b\"/\"openshift-service-ca.crt\""
Jul 29 16:28:23.286417 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.286393    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"]
Jul 29 16:28:23.417911 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.417878    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-tmp-dir\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.417911 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.417933    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-dshm\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.418151 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.417999    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-model-cache\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.418151 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.418051    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-home\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.418151 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.418072    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ll89\" (UniqueName: \"kubernetes.io/projected/7835f1ea-171a-4691-81ed-615aabd54e70-kube-api-access-2ll89\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.418151 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.418091    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7835f1ea-171a-4691-81ed-615aabd54e70-tls-certs\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.518796 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.518676    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-tmp-dir\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.518796 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.518736    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-dshm\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.518796 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.518772    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-model-cache\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.519150 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.518834    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-home\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.519150 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.518861    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ll89\" (UniqueName: \"kubernetes.io/projected/7835f1ea-171a-4691-81ed-615aabd54e70-kube-api-access-2ll89\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.519150 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.518888    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7835f1ea-171a-4691-81ed-615aabd54e70-tls-certs\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.519150 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.519121    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-tmp-dir\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.519362 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.519227    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-model-cache\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.519362 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.519276    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-home\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.521116 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.521093    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-dshm\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.521293 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.521275    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7835f1ea-171a-4691-81ed-615aabd54e70-tls-certs\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.527799 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.527782    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ll89\" (UniqueName: \"kubernetes.io/projected/7835f1ea-171a-4691-81ed-615aabd54e70-kube-api-access-2ll89\") pod \"auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") " pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.579393 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.579360    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:23.717629 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.717598    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"]
Jul 29 16:28:23.719251 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:28:23.719225    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7835f1ea_171a_4691_81ed_615aabd54e70.slice/crio-f333e72f674f1d4d0a6983082b9d460fbe5952703fce4d887646d895e34113e6 WatchSource:0}: Error finding container f333e72f674f1d4d0a6983082b9d460fbe5952703fce4d887646d895e34113e6: Status 404 returned error can't find the container with id f333e72f674f1d4d0a6983082b9d460fbe5952703fce4d887646d895e34113e6
Jul 29 16:28:23.881105 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.881063    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh" event={"ID":"7835f1ea-171a-4691-81ed-615aabd54e70","Type":"ContainerStarted","Data":"7c944053853c3cf499d4a73f572b456a9185026d492f5f47a0a26f74eda9c084"}
Jul 29 16:28:23.881105 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.881114    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh" event={"ID":"7835f1ea-171a-4691-81ed-615aabd54e70","Type":"ContainerStarted","Data":"f333e72f674f1d4d0a6983082b9d460fbe5952703fce4d887646d895e34113e6"}
Jul 29 16:28:23.900682 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:23.900627    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh" podStartSLOduration=0.900614617 podStartE2EDuration="900.614617ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:28:23.899307332 +0000 UTC m=+2508.341513591" watchObservedRunningTime="2026-07-29 16:28:23.900614617 +0000 UTC m=+2508.342820876"
Jul 29 16:28:26.974016 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:26.973981    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:28:33.579784 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:33.579683    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:33.579784 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:33.579765    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:33.592399 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:33.592377    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:33.940527 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:33.940501    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:28:45.659300 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.659268    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"]
Jul 29 16:28:45.662614 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.662597    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.666121 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.666097    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-eccd34\"/\"default-dockercfg-lwzzr\""
Jul 29 16:28:45.666249 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.666127    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-eccd34\"/\"openshift-service-ca.crt\""
Jul 29 16:28:45.667887 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.667871    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-eccd34\"/\"auth-test-model-eccd34-kserve-self-signed-certs\""
Jul 29 16:28:45.667990 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.667926    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-eccd34\"/\"kube-root-ca.crt\""
Jul 29 16:28:45.673440 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.673418    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"]
Jul 29 16:28:45.703074 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.703047    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-tmp-dir\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.703188 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.703094    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-model-cache\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.703188 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.703144    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43d13b3e-f8fc-4906-85db-2fb241783444-tls-certs\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.703274 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.703219    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-home\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.703274 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.703251    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m5l6\" (UniqueName: \"kubernetes.io/projected/43d13b3e-f8fc-4906-85db-2fb241783444-kube-api-access-6m5l6\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.703372 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.703275    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-dshm\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.804570 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.804537    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43d13b3e-f8fc-4906-85db-2fb241783444-tls-certs\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.804731 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.804579    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-home\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.804731 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.804602    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m5l6\" (UniqueName: \"kubernetes.io/projected/43d13b3e-f8fc-4906-85db-2fb241783444-kube-api-access-6m5l6\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.804731 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.804626    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-dshm\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.804731 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.804653    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-tmp-dir\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.804731 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.804682    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-model-cache\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.805063 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.805030    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-home\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.805142 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.805109    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-tmp-dir\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.805196 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.805143    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-model-cache\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.806930 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.806896    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-dshm\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.807128 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.807113    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43d13b3e-f8fc-4906-85db-2fb241783444-tls-certs\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.814535 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.814508    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m5l6\" (UniqueName: \"kubernetes.io/projected/43d13b3e-f8fc-4906-85db-2fb241783444-kube-api-access-6m5l6\") pod \"auth-test-model-eccd34-kserve-7bd4678479-kh44n\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") " pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:45.973844 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:45.973764    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:46.111374 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:46.111340    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"]
Jul 29 16:28:46.114392 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:28:46.114358    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43d13b3e_f8fc_4906_85db_2fb241783444.slice/crio-267b3532bdb021f8c2fe602141efd2ad532e683a8cd7bac0ebb430e311f2bfe9 WatchSource:0}: Error finding container 267b3532bdb021f8c2fe602141efd2ad532e683a8cd7bac0ebb430e311f2bfe9: Status 404 returned error can't find the container with id 267b3532bdb021f8c2fe602141efd2ad532e683a8cd7bac0ebb430e311f2bfe9
Jul 29 16:28:46.991580 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:46.991541    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n" event={"ID":"43d13b3e-f8fc-4906-85db-2fb241783444","Type":"ContainerStarted","Data":"7d7b6ea8ff2ee5c9c746b7088140a07b02d69f4b8dfe079a2b33cb08f6ab21d8"}
Jul 29 16:28:46.991580 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:46.991584    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n" event={"ID":"43d13b3e-f8fc-4906-85db-2fb241783444","Type":"ContainerStarted","Data":"267b3532bdb021f8c2fe602141efd2ad532e683a8cd7bac0ebb430e311f2bfe9"}
Jul 29 16:28:47.011268 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:47.011207    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n" podStartSLOduration=2.011188248 podStartE2EDuration="2.011188248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:28:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:28:47.009578425 +0000 UTC m=+2531.451784683" watchObservedRunningTime="2026-07-29 16:28:47.011188248 +0000 UTC m=+2531.453394508"
Jul 29 16:28:49.956670 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:49.956636    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:28:55.974514 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:55.974475    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:55.975159 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:55.974553    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:55.987018 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:55.986998    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:28:56.050755 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:28:56.050725    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:29:18.960908 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:29:18.960874    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:29:45.349793 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:29:45.349761    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:30:03.755456 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:30:03.755367    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:30:30.250587 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:30:30.250544    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:30:49.050256 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:30:49.050222    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:31:11.361556 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:31:11.361520    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:31:29.861633 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:31:29.861558    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:31:52.669367 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:31:52.669325    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:32:10.553623 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:32:10.553589    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:32:35.453187 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:32:35.453152    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:32:36.272628 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:32:36.272597    2560 scope.go:117] "RemoveContainer" containerID="e0d99d297bce66d509ec191bb5ebfbbffe63b775a647f5b0ca8feb83eca582c1"
Jul 29 16:32:54.561969 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:32:54.561925    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:33:14.953655 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:14.953553    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:33:19.721900 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:19.721867    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"]
Jul 29 16:33:20.654105 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:20.654075    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"]
Jul 29 16:33:32.057546 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:32.057511    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:33:34.740694 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:34.740634    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh" podUID="7835f1ea-171a-4691-81ed-615aabd54e70" containerName="main" containerID="cri-o://7c944053853c3cf499d4a73f572b456a9185026d492f5f47a0a26f74eda9c084" gracePeriod=45
Jul 29 16:33:34.991044 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:34.990986    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:33:35.138424 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.138392    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ll89\" (UniqueName: \"kubernetes.io/projected/7835f1ea-171a-4691-81ed-615aabd54e70-kube-api-access-2ll89\") pod \"7835f1ea-171a-4691-81ed-615aabd54e70\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") "
Jul 29 16:33:35.138424 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.138428    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7835f1ea-171a-4691-81ed-615aabd54e70-tls-certs\") pod \"7835f1ea-171a-4691-81ed-615aabd54e70\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") "
Jul 29 16:33:35.138691 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.138479    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-home\") pod \"7835f1ea-171a-4691-81ed-615aabd54e70\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") "
Jul 29 16:33:35.138691 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.138495    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-model-cache\") pod \"7835f1ea-171a-4691-81ed-615aabd54e70\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") "
Jul 29 16:33:35.138691 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.138521    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-tmp-dir\") pod \"7835f1ea-171a-4691-81ed-615aabd54e70\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") "
Jul 29 16:33:35.138691 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.138539    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-dshm\") pod \"7835f1ea-171a-4691-81ed-615aabd54e70\" (UID: \"7835f1ea-171a-4691-81ed-615aabd54e70\") "
Jul 29 16:33:35.138877 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.138757    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7835f1ea-171a-4691-81ed-615aabd54e70" (UID: "7835f1ea-171a-4691-81ed-615aabd54e70"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:33:35.138877 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.138778    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-model-cache" (OuterVolumeSpecName: "model-cache") pod "7835f1ea-171a-4691-81ed-615aabd54e70" (UID: "7835f1ea-171a-4691-81ed-615aabd54e70"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:33:35.138877 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.138787    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-home" (OuterVolumeSpecName: "home") pod "7835f1ea-171a-4691-81ed-615aabd54e70" (UID: "7835f1ea-171a-4691-81ed-615aabd54e70"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:33:35.140842 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.140817    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7835f1ea-171a-4691-81ed-615aabd54e70-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7835f1ea-171a-4691-81ed-615aabd54e70" (UID: "7835f1ea-171a-4691-81ed-615aabd54e70"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:33:35.140842 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.140828    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-dshm" (OuterVolumeSpecName: "dshm") pod "7835f1ea-171a-4691-81ed-615aabd54e70" (UID: "7835f1ea-171a-4691-81ed-615aabd54e70"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:33:35.140982 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.140844    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7835f1ea-171a-4691-81ed-615aabd54e70-kube-api-access-2ll89" (OuterVolumeSpecName: "kube-api-access-2ll89") pod "7835f1ea-171a-4691-81ed-615aabd54e70" (UID: "7835f1ea-171a-4691-81ed-615aabd54e70"). InnerVolumeSpecName "kube-api-access-2ll89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:33:35.240138 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.240108    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-tmp-dir\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:33:35.240138 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.240131    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-dshm\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:33:35.240138 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.240143    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ll89\" (UniqueName: \"kubernetes.io/projected/7835f1ea-171a-4691-81ed-615aabd54e70-kube-api-access-2ll89\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:33:35.240343 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.240152    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7835f1ea-171a-4691-81ed-615aabd54e70-tls-certs\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:33:35.240343 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.240162    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-home\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:33:35.240343 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.240170    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7835f1ea-171a-4691-81ed-615aabd54e70-model-cache\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:33:35.400908 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.400871    2560 generic.go:356] "Generic (PLEG): container finished" podID="7835f1ea-171a-4691-81ed-615aabd54e70" containerID="7c944053853c3cf499d4a73f572b456a9185026d492f5f47a0a26f74eda9c084" exitCode=0
Jul 29 16:33:35.401071 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.400940    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"
Jul 29 16:33:35.401071 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.400958    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh" event={"ID":"7835f1ea-171a-4691-81ed-615aabd54e70","Type":"ContainerDied","Data":"7c944053853c3cf499d4a73f572b456a9185026d492f5f47a0a26f74eda9c084"}
Jul 29 16:33:35.401071 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.401004    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh" event={"ID":"7835f1ea-171a-4691-81ed-615aabd54e70","Type":"ContainerDied","Data":"f333e72f674f1d4d0a6983082b9d460fbe5952703fce4d887646d895e34113e6"}
Jul 29 16:33:35.401071 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.401020    2560 scope.go:117] "RemoveContainer" containerID="7c944053853c3cf499d4a73f572b456a9185026d492f5f47a0a26f74eda9c084"
Jul 29 16:33:35.410045 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.410027    2560 scope.go:117] "RemoveContainer" containerID="7c944053853c3cf499d4a73f572b456a9185026d492f5f47a0a26f74eda9c084"
Jul 29 16:33:35.410282 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:33:35.410262    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c944053853c3cf499d4a73f572b456a9185026d492f5f47a0a26f74eda9c084\": container with ID starting with 7c944053853c3cf499d4a73f572b456a9185026d492f5f47a0a26f74eda9c084 not found: ID does not exist" containerID="7c944053853c3cf499d4a73f572b456a9185026d492f5f47a0a26f74eda9c084"
Jul 29 16:33:35.410335 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.410291    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c944053853c3cf499d4a73f572b456a9185026d492f5f47a0a26f74eda9c084"} err="failed to get container status \"7c944053853c3cf499d4a73f572b456a9185026d492f5f47a0a26f74eda9c084\": rpc error: code = NotFound desc = could not find container \"7c944053853c3cf499d4a73f572b456a9185026d492f5f47a0a26f74eda9c084\": container with ID starting with 7c944053853c3cf499d4a73f572b456a9185026d492f5f47a0a26f74eda9c084 not found: ID does not exist"
Jul 29 16:33:35.424148 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.424120    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"]
Jul 29 16:33:35.425856 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.425837    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-81ab8b/auth-test-model-81ab8b-kserve-7f988b89f5-tx6sh"]
Jul 29 16:33:35.673264 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.673176    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n" podUID="43d13b3e-f8fc-4906-85db-2fb241783444" containerName="main" containerID="cri-o://7d7b6ea8ff2ee5c9c746b7088140a07b02d69f4b8dfe079a2b33cb08f6ab21d8" gracePeriod=45
Jul 29 16:33:35.923899 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:35.923847    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:33:36.043261 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.043228    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7835f1ea-171a-4691-81ed-615aabd54e70" path="/var/lib/kubelet/pods/7835f1ea-171a-4691-81ed-615aabd54e70/volumes"
Jul 29 16:33:36.048156 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.048132    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43d13b3e-f8fc-4906-85db-2fb241783444-tls-certs\") pod \"43d13b3e-f8fc-4906-85db-2fb241783444\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") "
Jul 29 16:33:36.048316 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.048170    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-model-cache\") pod \"43d13b3e-f8fc-4906-85db-2fb241783444\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") "
Jul 29 16:33:36.048316 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.048208    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m5l6\" (UniqueName: \"kubernetes.io/projected/43d13b3e-f8fc-4906-85db-2fb241783444-kube-api-access-6m5l6\") pod \"43d13b3e-f8fc-4906-85db-2fb241783444\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") "
Jul 29 16:33:36.048316 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.048248    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-tmp-dir\") pod \"43d13b3e-f8fc-4906-85db-2fb241783444\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") "
Jul 29 16:33:36.048316 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.048277    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-home\") pod \"43d13b3e-f8fc-4906-85db-2fb241783444\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") "
Jul 29 16:33:36.048316 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.048302    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-dshm\") pod \"43d13b3e-f8fc-4906-85db-2fb241783444\" (UID: \"43d13b3e-f8fc-4906-85db-2fb241783444\") "
Jul 29 16:33:36.048610 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.048444    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-model-cache" (OuterVolumeSpecName: "model-cache") pod "43d13b3e-f8fc-4906-85db-2fb241783444" (UID: "43d13b3e-f8fc-4906-85db-2fb241783444"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:33:36.048610 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.048523    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-model-cache\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:33:36.048610 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.048531    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-home" (OuterVolumeSpecName: "home") pod "43d13b3e-f8fc-4906-85db-2fb241783444" (UID: "43d13b3e-f8fc-4906-85db-2fb241783444"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:33:36.048610 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.048538    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "43d13b3e-f8fc-4906-85db-2fb241783444" (UID: "43d13b3e-f8fc-4906-85db-2fb241783444"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:33:36.050317 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.050293    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-dshm" (OuterVolumeSpecName: "dshm") pod "43d13b3e-f8fc-4906-85db-2fb241783444" (UID: "43d13b3e-f8fc-4906-85db-2fb241783444"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:33:36.050409 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.050373    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d13b3e-f8fc-4906-85db-2fb241783444-kube-api-access-6m5l6" (OuterVolumeSpecName: "kube-api-access-6m5l6") pod "43d13b3e-f8fc-4906-85db-2fb241783444" (UID: "43d13b3e-f8fc-4906-85db-2fb241783444"). InnerVolumeSpecName "kube-api-access-6m5l6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:33:36.050409 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.050379    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43d13b3e-f8fc-4906-85db-2fb241783444-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "43d13b3e-f8fc-4906-85db-2fb241783444" (UID: "43d13b3e-f8fc-4906-85db-2fb241783444"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:33:36.149151 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.149117    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-tmp-dir\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:33:36.149151 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.149142    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-home\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:33:36.149151 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.149150    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43d13b3e-f8fc-4906-85db-2fb241783444-dshm\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:33:36.149151 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.149158    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43d13b3e-f8fc-4906-85db-2fb241783444-tls-certs\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:33:36.149397 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.149166    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m5l6\" (UniqueName: \"kubernetes.io/projected/43d13b3e-f8fc-4906-85db-2fb241783444-kube-api-access-6m5l6\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:33:36.406957 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.406901    2560 generic.go:356] "Generic (PLEG): container finished" podID="43d13b3e-f8fc-4906-85db-2fb241783444" containerID="7d7b6ea8ff2ee5c9c746b7088140a07b02d69f4b8dfe079a2b33cb08f6ab21d8" exitCode=0
Jul 29 16:33:36.406957 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.406958    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"
Jul 29 16:33:36.407196 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.406990    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n" event={"ID":"43d13b3e-f8fc-4906-85db-2fb241783444","Type":"ContainerDied","Data":"7d7b6ea8ff2ee5c9c746b7088140a07b02d69f4b8dfe079a2b33cb08f6ab21d8"}
Jul 29 16:33:36.407196 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.407030    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n" event={"ID":"43d13b3e-f8fc-4906-85db-2fb241783444","Type":"ContainerDied","Data":"267b3532bdb021f8c2fe602141efd2ad532e683a8cd7bac0ebb430e311f2bfe9"}
Jul 29 16:33:36.407196 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.407054    2560 scope.go:117] "RemoveContainer" containerID="7d7b6ea8ff2ee5c9c746b7088140a07b02d69f4b8dfe079a2b33cb08f6ab21d8"
Jul 29 16:33:36.416481 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.416459    2560 scope.go:117] "RemoveContainer" containerID="7d7b6ea8ff2ee5c9c746b7088140a07b02d69f4b8dfe079a2b33cb08f6ab21d8"
Jul 29 16:33:36.416729 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:33:36.416709    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d7b6ea8ff2ee5c9c746b7088140a07b02d69f4b8dfe079a2b33cb08f6ab21d8\": container with ID starting with 7d7b6ea8ff2ee5c9c746b7088140a07b02d69f4b8dfe079a2b33cb08f6ab21d8 not found: ID does not exist" containerID="7d7b6ea8ff2ee5c9c746b7088140a07b02d69f4b8dfe079a2b33cb08f6ab21d8"
Jul 29 16:33:36.416794 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.416736    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7b6ea8ff2ee5c9c746b7088140a07b02d69f4b8dfe079a2b33cb08f6ab21d8"} err="failed to get container status \"7d7b6ea8ff2ee5c9c746b7088140a07b02d69f4b8dfe079a2b33cb08f6ab21d8\": rpc error: code = NotFound desc = could not find container \"7d7b6ea8ff2ee5c9c746b7088140a07b02d69f4b8dfe079a2b33cb08f6ab21d8\": container with ID starting with 7d7b6ea8ff2ee5c9c746b7088140a07b02d69f4b8dfe079a2b33cb08f6ab21d8 not found: ID does not exist"
Jul 29 16:33:36.431049 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.431024    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"]
Jul 29 16:33:36.435048 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:36.435027    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/auth-test-model-eccd34-kserve-7bd4678479-kh44n"]
Jul 29 16:33:38.043045 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:38.043012    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43d13b3e-f8fc-4906-85db-2fb241783444" path="/var/lib/kubelet/pods/43d13b3e-f8fc-4906-85db-2fb241783444/volumes"
Jul 29 16:33:44.361697 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:33:44.361667    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:34:03.659655 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.659619    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"]
Jul 29 16:34:03.660131 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.659993    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7835f1ea-171a-4691-81ed-615aabd54e70" containerName="main"
Jul 29 16:34:03.660131 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.660005    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7835f1ea-171a-4691-81ed-615aabd54e70" containerName="main"
Jul 29 16:34:03.660131 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.660051    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43d13b3e-f8fc-4906-85db-2fb241783444" containerName="main"
Jul 29 16:34:03.660131 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.660056    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="43d13b3e-f8fc-4906-85db-2fb241783444" containerName="main"
Jul 29 16:34:03.660131 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.660121    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="43d13b3e-f8fc-4906-85db-2fb241783444" containerName="main"
Jul 29 16:34:03.660335 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.660139    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7835f1ea-171a-4691-81ed-615aabd54e70" containerName="main"
Jul 29 16:34:03.663235 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.663215    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.666293 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.666273    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-eccd34\"/\"kube-root-ca.crt\""
Jul 29 16:34:03.667509 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.667489    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-eccd34\"/\"sub-test-model-eccd34-kserve-self-signed-certs\""
Jul 29 16:34:03.667624 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.667520    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-eccd34\"/\"openshift-service-ca.crt\""
Jul 29 16:34:03.667624 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.667530    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-eccd34\"/\"default-dockercfg-lwzzr\""
Jul 29 16:34:03.672963 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.672941    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"]
Jul 29 16:34:03.788827 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.788776    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-dshm\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.788827 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.788829    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-home\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.789074 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.788853    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-model-cache\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.789074 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.788868    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-tmp-dir\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.789074 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.788998    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a69ccad-6357-4606-bed1-c29de498136a-tls-certs\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.789074 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.789029    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft7wz\" (UniqueName: \"kubernetes.io/projected/9a69ccad-6357-4606-bed1-c29de498136a-kube-api-access-ft7wz\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.890313 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.890261    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a69ccad-6357-4606-bed1-c29de498136a-tls-certs\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.890313 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.890314    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft7wz\" (UniqueName: \"kubernetes.io/projected/9a69ccad-6357-4606-bed1-c29de498136a-kube-api-access-ft7wz\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.890579 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.890361    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-dshm\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.890579 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.890380    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-home\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.890579 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.890397    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-model-cache\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.890579 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.890414    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-tmp-dir\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.890864 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.890841    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-home\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.890977 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.890889    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-tmp-dir\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.890977 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.890946    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-model-cache\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.892584 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.892567    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-dshm\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.892877 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.892861    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a69ccad-6357-4606-bed1-c29de498136a-tls-certs\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.899259 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.899239    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft7wz\" (UniqueName: \"kubernetes.io/projected/9a69ccad-6357-4606-bed1-c29de498136a-kube-api-access-ft7wz\") pod \"sub-test-model-eccd34-kserve-669ff55c5f-qt97d\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") " pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:03.971860 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:03.971766    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:04.109506 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:04.109473    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"]
Jul 29 16:34:04.112879 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:34:04.112850    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a69ccad_6357_4606_bed1_c29de498136a.slice/crio-c964832c9a6cf490e5e6aa4c22d51949ab5b4bc9258a0775a2032d5e2561157d WatchSource:0}: Error finding container c964832c9a6cf490e5e6aa4c22d51949ab5b4bc9258a0775a2032d5e2561157d: Status 404 returned error can't find the container with id c964832c9a6cf490e5e6aa4c22d51949ab5b4bc9258a0775a2032d5e2561157d
Jul 29 16:34:04.541847 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:04.541808    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d" event={"ID":"9a69ccad-6357-4606-bed1-c29de498136a","Type":"ContainerStarted","Data":"1649af6b43b5c8638c321e5799dc4d9a14f7587ea5eb166294353763bc78bb6f"}
Jul 29 16:34:04.542053 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:04.541856    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d" event={"ID":"9a69ccad-6357-4606-bed1-c29de498136a","Type":"ContainerStarted","Data":"c964832c9a6cf490e5e6aa4c22d51949ab5b4bc9258a0775a2032d5e2561157d"}
Jul 29 16:34:04.561442 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:04.561398    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d" podStartSLOduration=1.561382222 podStartE2EDuration="1.561382222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:34:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:34:04.560743774 +0000 UTC m=+2849.002950044" watchObservedRunningTime="2026-07-29 16:34:04.561382222 +0000 UTC m=+2849.003588482"
Jul 29 16:34:07.454744 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:07.454705    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:34:13.972992 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:13.972955    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:13.973400 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:13.973009    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:13.985198 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:13.985164    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:14.601211 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:14.601179    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:34:39.555424 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:39.555383    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:34:59.257666 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:34:59.257632    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:35:06.658309 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:06.658270    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"]
Jul 29 16:35:16.663958 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.663907    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"]
Jul 29 16:35:16.667337 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.667316    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.670424 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.670401    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-81ab8b\"/\"kube-root-ca.crt\""
Jul 29 16:35:16.671580 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.671563    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-81ab8b\"/\"default-dockercfg-knk2z\""
Jul 29 16:35:16.671680 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.671658    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-81ab8b\"/\"openshift-service-ca.crt\""
Jul 29 16:35:16.671680 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.671660    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-81ab8b\"/\"rate-test-model-81ab8b-kserve-self-signed-certs\""
Jul 29 16:35:16.679014 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.678993    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"]
Jul 29 16:35:16.812080 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.812048    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxb87\" (UniqueName: \"kubernetes.io/projected/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-kube-api-access-wxb87\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.812080 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.812082    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-tls-certs\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.812300 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.812103    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-tmp-dir\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.812300 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.812241    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-home\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.812300 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.812289    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-model-cache\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.812419 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.812345    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-dshm\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.913293 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.913259    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-model-cache\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.913442 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.913298    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-dshm\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.913442 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.913330    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxb87\" (UniqueName: \"kubernetes.io/projected/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-kube-api-access-wxb87\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.913442 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.913356    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-tls-certs\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.913442 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.913382    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-tmp-dir\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.913682 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.913494    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-home\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.913745 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.913703    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-model-cache\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.913809 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.913782    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-tmp-dir\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.913866 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.913830    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-home\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.915543 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.915488    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-dshm\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.915924 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.915892    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-tls-certs\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.921872 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.921855    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxb87\" (UniqueName: \"kubernetes.io/projected/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-kube-api-access-wxb87\") pod \"rate-test-model-81ab8b-kserve-8489f8b499-fzkcf\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") " pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:16.978341 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:16.978318    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:17.118371 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:35:17.118330    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3f3204c_ef7a_4f07_a0db_e5f4a8b1453b.slice/crio-0a1d5eada0d82998e01703b8743cee9bb6e9636c6201659f319201d97b7cd846 WatchSource:0}: Error finding container 0a1d5eada0d82998e01703b8743cee9bb6e9636c6201659f319201d97b7cd846: Status 404 returned error can't find the container with id 0a1d5eada0d82998e01703b8743cee9bb6e9636c6201659f319201d97b7cd846
Jul 29 16:35:17.118826 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:17.118743    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"]
Jul 29 16:35:17.898688 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:17.898653    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf" event={"ID":"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b","Type":"ContainerStarted","Data":"f168623085f75c2154f79a3bbbdcbcae051b1ff8a708207be7d056aaf16bf930"}
Jul 29 16:35:17.898688 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:17.898692    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf" event={"ID":"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b","Type":"ContainerStarted","Data":"0a1d5eada0d82998e01703b8743cee9bb6e9636c6201659f319201d97b7cd846"}
Jul 29 16:35:17.925279 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:17.925231    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf" podStartSLOduration=1.925217898 podStartE2EDuration="1.925217898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:35:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:35:17.923283541 +0000 UTC m=+2922.365489801" watchObservedRunningTime="2026-07-29 16:35:17.925217898 +0000 UTC m=+2922.367424190"
Jul 29 16:35:19.565711 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:19.565675    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:35:21.676862 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:21.676823    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d" podUID="9a69ccad-6357-4606-bed1-c29de498136a" containerName="main" containerID="cri-o://1649af6b43b5c8638c321e5799dc4d9a14f7587ea5eb166294353763bc78bb6f" gracePeriod=45
Jul 29 16:35:21.922225 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:21.922198    2560 generic.go:356] "Generic (PLEG): container finished" podID="9a69ccad-6357-4606-bed1-c29de498136a" containerID="1649af6b43b5c8638c321e5799dc4d9a14f7587ea5eb166294353763bc78bb6f" exitCode=0
Jul 29 16:35:21.922369 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:21.922270    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d" event={"ID":"9a69ccad-6357-4606-bed1-c29de498136a","Type":"ContainerDied","Data":"1649af6b43b5c8638c321e5799dc4d9a14f7587ea5eb166294353763bc78bb6f"}
Jul 29 16:35:21.922369 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:21.922308    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d" event={"ID":"9a69ccad-6357-4606-bed1-c29de498136a","Type":"ContainerDied","Data":"c964832c9a6cf490e5e6aa4c22d51949ab5b4bc9258a0775a2032d5e2561157d"}
Jul 29 16:35:21.922369 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:21.922318    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c964832c9a6cf490e5e6aa4c22d51949ab5b4bc9258a0775a2032d5e2561157d"
Jul 29 16:35:21.931549 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:21.931495    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:35:22.059273 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.059244    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-dshm\") pod \"9a69ccad-6357-4606-bed1-c29de498136a\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") "
Jul 29 16:35:22.059448 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.059334    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a69ccad-6357-4606-bed1-c29de498136a-tls-certs\") pod \"9a69ccad-6357-4606-bed1-c29de498136a\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") "
Jul 29 16:35:22.059448 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.059355    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ft7wz\" (UniqueName: \"kubernetes.io/projected/9a69ccad-6357-4606-bed1-c29de498136a-kube-api-access-ft7wz\") pod \"9a69ccad-6357-4606-bed1-c29de498136a\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") "
Jul 29 16:35:22.059448 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.059383    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-home\") pod \"9a69ccad-6357-4606-bed1-c29de498136a\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") "
Jul 29 16:35:22.059448 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.059427    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-model-cache\") pod \"9a69ccad-6357-4606-bed1-c29de498136a\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") "
Jul 29 16:35:22.059678 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.059505    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-tmp-dir\") pod \"9a69ccad-6357-4606-bed1-c29de498136a\" (UID: \"9a69ccad-6357-4606-bed1-c29de498136a\") "
Jul 29 16:35:22.059678 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.059669    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-home" (OuterVolumeSpecName: "home") pod "9a69ccad-6357-4606-bed1-c29de498136a" (UID: "9a69ccad-6357-4606-bed1-c29de498136a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:35:22.059787 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.059683    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-model-cache" (OuterVolumeSpecName: "model-cache") pod "9a69ccad-6357-4606-bed1-c29de498136a" (UID: "9a69ccad-6357-4606-bed1-c29de498136a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:35:22.059787 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.059748    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9a69ccad-6357-4606-bed1-c29de498136a" (UID: "9a69ccad-6357-4606-bed1-c29de498136a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:35:22.059893 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.059844    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-home\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:35:22.059893 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.059860    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-model-cache\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:35:22.059893 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.059869    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-tmp-dir\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:35:22.061481 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.061460    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-dshm" (OuterVolumeSpecName: "dshm") pod "9a69ccad-6357-4606-bed1-c29de498136a" (UID: "9a69ccad-6357-4606-bed1-c29de498136a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:35:22.061585 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.061459    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a69ccad-6357-4606-bed1-c29de498136a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9a69ccad-6357-4606-bed1-c29de498136a" (UID: "9a69ccad-6357-4606-bed1-c29de498136a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:35:22.061585 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.061540    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a69ccad-6357-4606-bed1-c29de498136a-kube-api-access-ft7wz" (OuterVolumeSpecName: "kube-api-access-ft7wz") pod "9a69ccad-6357-4606-bed1-c29de498136a" (UID: "9a69ccad-6357-4606-bed1-c29de498136a"). InnerVolumeSpecName "kube-api-access-ft7wz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:35:22.160435 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.160407    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft7wz\" (UniqueName: \"kubernetes.io/projected/9a69ccad-6357-4606-bed1-c29de498136a-kube-api-access-ft7wz\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:35:22.160435 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.160428    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a69ccad-6357-4606-bed1-c29de498136a-dshm\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:35:22.160435 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.160438    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a69ccad-6357-4606-bed1-c29de498136a-tls-certs\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:35:22.929948 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.929085    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"
Jul 29 16:35:22.954720 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.954693    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"]
Jul 29 16:35:22.956411 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:22.956391    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/sub-test-model-eccd34-kserve-669ff55c5f-qt97d"]
Jul 29 16:35:24.042832 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:24.042796    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a69ccad-6357-4606-bed1-c29de498136a" path="/var/lib/kubelet/pods/9a69ccad-6357-4606-bed1-c29de498136a/volumes"
Jul 29 16:35:26.978871 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:26.978842    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:26.979359 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:26.978897    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:26.991011 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:26.990990    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:27.964730 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:27.964696    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:35:54.474769 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.474738    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"]
Jul 29 16:35:54.475244 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.475119    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a69ccad-6357-4606-bed1-c29de498136a" containerName="main"
Jul 29 16:35:54.475244 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.475131    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a69ccad-6357-4606-bed1-c29de498136a" containerName="main"
Jul 29 16:35:54.475244 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.475188    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a69ccad-6357-4606-bed1-c29de498136a" containerName="main"
Jul 29 16:35:54.478226 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.478205    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.481099 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.481072    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-eccd34\"/\"default-dockercfg-lwzzr\""
Jul 29 16:35:54.481186 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.481104    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-eccd34\"/\"kube-root-ca.crt\""
Jul 29 16:35:54.482275 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.482253    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-eccd34\"/\"openshift-service-ca.crt\""
Jul 29 16:35:54.482379 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.482347    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-eccd34\"/\"rate-test-model-eccd34-kserve-self-signed-certs\""
Jul 29 16:35:54.487888 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.487863    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"]
Jul 29 16:35:54.546291 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.546257    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-model-cache\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.546461 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.546348    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-tmp-dir\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.546461 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.546376    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-home\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.546461 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.546394    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s9xl\" (UniqueName: \"kubernetes.io/projected/981e18f4-ffed-43a9-a86a-123fd6772b64-kube-api-access-6s9xl\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.546461 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.546428    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-dshm\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.546461 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.546456    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/981e18f4-ffed-43a9-a86a-123fd6772b64-tls-certs\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.647831 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.647796    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-model-cache\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.648021 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.647874    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-tmp-dir\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.648021 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.647906    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-home\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.648021 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.647946    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s9xl\" (UniqueName: \"kubernetes.io/projected/981e18f4-ffed-43a9-a86a-123fd6772b64-kube-api-access-6s9xl\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.648021 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.647984    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-dshm\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.648179 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.648022    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/981e18f4-ffed-43a9-a86a-123fd6772b64-tls-certs\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.648324 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.648302    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-model-cache\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.648373 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.648334    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-tmp-dir\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.648409 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.648368    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-home\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.650370 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.650345    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-dshm\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.650594 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.650577    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/981e18f4-ffed-43a9-a86a-123fd6772b64-tls-certs\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.656120 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.656100    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s9xl\" (UniqueName: \"kubernetes.io/projected/981e18f4-ffed-43a9-a86a-123fd6772b64-kube-api-access-6s9xl\") pod \"rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") " pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.787504 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.787407    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:35:54.919355 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:54.919312    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"]
Jul 29 16:35:54.922229 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:35:54.922199    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod981e18f4_ffed_43a9_a86a_123fd6772b64.slice/crio-ec79a58f649bce328b73ed6bfb19d8e49c05f615d0850de6e499dc0ba11e1486 WatchSource:0}: Error finding container ec79a58f649bce328b73ed6bfb19d8e49c05f615d0850de6e499dc0ba11e1486: Status 404 returned error can't find the container with id ec79a58f649bce328b73ed6bfb19d8e49c05f615d0850de6e499dc0ba11e1486
Jul 29 16:35:55.089689 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:55.089584    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt" event={"ID":"981e18f4-ffed-43a9-a86a-123fd6772b64","Type":"ContainerStarted","Data":"dc7d7cc6d81ed5f3cb83756d1ccb114b3f42dca5648761009e7288bb2cb5728c"}
Jul 29 16:35:55.089689 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:55.089626    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt" event={"ID":"981e18f4-ffed-43a9-a86a-123fd6772b64","Type":"ContainerStarted","Data":"ec79a58f649bce328b73ed6bfb19d8e49c05f615d0850de6e499dc0ba11e1486"}
Jul 29 16:35:55.108154 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:55.108099    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt" podStartSLOduration=1.108082917 podStartE2EDuration="1.108082917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:35:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:35:55.106988938 +0000 UTC m=+2959.549195198" watchObservedRunningTime="2026-07-29 16:35:55.108082917 +0000 UTC m=+2959.550289154"
Jul 29 16:35:58.255164 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:35:58.255072    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:36:04.787794 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:04.787765    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:36:04.787794 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:04.787804    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:36:04.800400 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:04.800376    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:36:05.150500 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:05.150473    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:36:30.649458 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:30.649416    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:36:51.155923 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:51.155892    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:36:57.467789 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:57.467756    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"]
Jul 29 16:36:58.396580 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:58.396543    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"]
Jul 29 16:36:59.366242 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:59.366209    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7654-139973292215344"]
Jul 29 16:36:59.369646 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:59.369628    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-139973292215344"
Jul 29 16:36:59.372388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:59.372357    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lqpzd\""
Jul 29 16:36:59.376890 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:59.376559    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7654-139973292215344"]
Jul 29 16:36:59.507040 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:59.507004    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7rpd\" (UniqueName: \"kubernetes.io/projected/0eb2b3cb-60c5-49df-9d62-3fa60644caba-kube-api-access-l7rpd\") pod \"test-curl-7654-139973292215344\" (UID: \"0eb2b3cb-60c5-49df-9d62-3fa60644caba\") " pod="opendatahub/test-curl-7654-139973292215344"
Jul 29 16:36:59.608435 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:59.608405    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7rpd\" (UniqueName: \"kubernetes.io/projected/0eb2b3cb-60c5-49df-9d62-3fa60644caba-kube-api-access-l7rpd\") pod \"test-curl-7654-139973292215344\" (UID: \"0eb2b3cb-60c5-49df-9d62-3fa60644caba\") " pod="opendatahub/test-curl-7654-139973292215344"
Jul 29 16:36:59.618697 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:59.618643    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7rpd\" (UniqueName: \"kubernetes.io/projected/0eb2b3cb-60c5-49df-9d62-3fa60644caba-kube-api-access-l7rpd\") pod \"test-curl-7654-139973292215344\" (UID: \"0eb2b3cb-60c5-49df-9d62-3fa60644caba\") " pod="opendatahub/test-curl-7654-139973292215344"
Jul 29 16:36:59.679793 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:59.679766    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-139973292215344"
Jul 29 16:36:59.821273 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:59.821246    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7654-139973292215344"]
Jul 29 16:36:59.823693 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:36:59.823664    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0eb2b3cb_60c5_49df_9d62_3fa60644caba.slice/crio-5b1560b5fcffef6d0a2a31ec75a205d35c5de57cf5809943e8c38a94b68095ab WatchSource:0}: Error finding container 5b1560b5fcffef6d0a2a31ec75a205d35c5de57cf5809943e8c38a94b68095ab: Status 404 returned error can't find the container with id 5b1560b5fcffef6d0a2a31ec75a205d35c5de57cf5809943e8c38a94b68095ab
Jul 29 16:36:59.825055 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:36:59.825037    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 16:37:00.415807 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:00.415772    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7654-139973292215344" event={"ID":"0eb2b3cb-60c5-49df-9d62-3fa60644caba","Type":"ContainerStarted","Data":"5b1560b5fcffef6d0a2a31ec75a205d35c5de57cf5809943e8c38a94b68095ab"}
Jul 29 16:37:01.422002 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:01.421967    2560 generic.go:356] "Generic (PLEG): container finished" podID="0eb2b3cb-60c5-49df-9d62-3fa60644caba" containerID="574c669a6f209dc84dc66e4fde27cc2f2c310c7cc7d5ae3aad1901f82af1d250" exitCode=0
Jul 29 16:37:01.422392 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:01.422053    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7654-139973292215344" event={"ID":"0eb2b3cb-60c5-49df-9d62-3fa60644caba","Type":"ContainerDied","Data":"574c669a6f209dc84dc66e4fde27cc2f2c310c7cc7d5ae3aad1901f82af1d250"}
Jul 29 16:37:02.561387 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:02.561361    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-139973292215344"
Jul 29 16:37:02.600855 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:02.600822    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7654-139973292215344_0eb2b3cb-60c5-49df-9d62-3fa60644caba/test-curl-7654-139973292215344/0.log"
Jul 29 16:37:02.624480 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:02.624443    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7654-139973292215344"]
Jul 29 16:37:02.628127 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:02.628098    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7654-139973292215344"]
Jul 29 16:37:02.738387 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:02.738299    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7rpd\" (UniqueName: \"kubernetes.io/projected/0eb2b3cb-60c5-49df-9d62-3fa60644caba-kube-api-access-l7rpd\") pod \"0eb2b3cb-60c5-49df-9d62-3fa60644caba\" (UID: \"0eb2b3cb-60c5-49df-9d62-3fa60644caba\") "
Jul 29 16:37:02.740520 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:02.740485    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eb2b3cb-60c5-49df-9d62-3fa60644caba-kube-api-access-l7rpd" (OuterVolumeSpecName: "kube-api-access-l7rpd") pod "0eb2b3cb-60c5-49df-9d62-3fa60644caba" (UID: "0eb2b3cb-60c5-49df-9d62-3fa60644caba"). InnerVolumeSpecName "kube-api-access-l7rpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:37:02.839223 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:02.839185    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7rpd\" (UniqueName: \"kubernetes.io/projected/0eb2b3cb-60c5-49df-9d62-3fa60644caba-kube-api-access-l7rpd\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:03.433354 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:03.433331    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-139973292215344"
Jul 29 16:37:03.433527 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:03.433369    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b1560b5fcffef6d0a2a31ec75a205d35c5de57cf5809943e8c38a94b68095ab"
Jul 29 16:37:03.446951 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:37:03.446901    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7654-139973292215344\" is forbidden: User \"system:node:ip-10-0-143-118.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-118.ec2.internal' and this object" podUID="0eb2b3cb-60c5-49df-9d62-3fa60644caba" pod="opendatahub/test-curl-7654-139973292215344"
Jul 29 16:37:04.043630 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:04.043594    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0eb2b3cb-60c5-49df-9d62-3fa60644caba" path="/var/lib/kubelet/pods/0eb2b3cb-60c5-49df-9d62-3fa60644caba/volumes"
Jul 29 16:37:05.739092 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:05.739061    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7654-139973292300304"]
Jul 29 16:37:05.739467 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:05.739417    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0eb2b3cb-60c5-49df-9d62-3fa60644caba" containerName="test-curl-7654-139973292215344"
Jul 29 16:37:05.739467 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:05.739428    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0eb2b3cb-60c5-49df-9d62-3fa60644caba" containerName="test-curl-7654-139973292215344"
Jul 29 16:37:05.739544 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:05.739505    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0eb2b3cb-60c5-49df-9d62-3fa60644caba" containerName="test-curl-7654-139973292215344"
Jul 29 16:37:05.742666 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:05.742649    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-139973292300304"
Jul 29 16:37:05.746518 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:05.746488    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lqpzd\""
Jul 29 16:37:05.749976 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:05.749688    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7654-139973292300304"]
Jul 29 16:37:05.866603 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:05.866571    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t42tv\" (UniqueName: \"kubernetes.io/projected/35ae8a64-5ef5-4609-9647-38766deab1e0-kube-api-access-t42tv\") pod \"test-curl-7654-139973292300304\" (UID: \"35ae8a64-5ef5-4609-9647-38766deab1e0\") " pod="opendatahub/test-curl-7654-139973292300304"
Jul 29 16:37:05.967985 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:05.967954    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t42tv\" (UniqueName: \"kubernetes.io/projected/35ae8a64-5ef5-4609-9647-38766deab1e0-kube-api-access-t42tv\") pod \"test-curl-7654-139973292300304\" (UID: \"35ae8a64-5ef5-4609-9647-38766deab1e0\") " pod="opendatahub/test-curl-7654-139973292300304"
Jul 29 16:37:05.976880 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:05.976858    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t42tv\" (UniqueName: \"kubernetes.io/projected/35ae8a64-5ef5-4609-9647-38766deab1e0-kube-api-access-t42tv\") pod \"test-curl-7654-139973292300304\" (UID: \"35ae8a64-5ef5-4609-9647-38766deab1e0\") " pod="opendatahub/test-curl-7654-139973292300304"
Jul 29 16:37:06.052519 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:06.052458    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-139973292300304"
Jul 29 16:37:06.393089 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:06.393065    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7654-139973292300304"]
Jul 29 16:37:06.395304 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:37:06.395275    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35ae8a64_5ef5_4609_9647_38766deab1e0.slice/crio-a950fedd6d28f616ffa29cbe9f4b68680ce86756271c35e7533d94f22dcb4211 WatchSource:0}: Error finding container a950fedd6d28f616ffa29cbe9f4b68680ce86756271c35e7533d94f22dcb4211: Status 404 returned error can't find the container with id a950fedd6d28f616ffa29cbe9f4b68680ce86756271c35e7533d94f22dcb4211
Jul 29 16:37:06.449834 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:06.449808    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7654-139973292300304" event={"ID":"35ae8a64-5ef5-4609-9647-38766deab1e0","Type":"ContainerStarted","Data":"a950fedd6d28f616ffa29cbe9f4b68680ce86756271c35e7533d94f22dcb4211"}
Jul 29 16:37:07.456287 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:07.456255    2560 generic.go:356] "Generic (PLEG): container finished" podID="35ae8a64-5ef5-4609-9647-38766deab1e0" containerID="0035564813d0f0ba0a73753aefd776ea0cf5b4966e618d6d16ba364728bd8aec" exitCode=0
Jul 29 16:37:07.456674 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:07.456342    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7654-139973292300304" event={"ID":"35ae8a64-5ef5-4609-9647-38766deab1e0","Type":"ContainerDied","Data":"0035564813d0f0ba0a73753aefd776ea0cf5b4966e618d6d16ba364728bd8aec"}
Jul 29 16:37:08.589416 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:08.589388    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-139973292300304"
Jul 29 16:37:08.611885 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:08.611860    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7654-139973292300304_35ae8a64-5ef5-4609-9647-38766deab1e0/test-curl-7654-139973292300304/0.log"
Jul 29 16:37:08.637456 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:08.637426    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7654-139973292300304"]
Jul 29 16:37:08.643443 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:08.643418    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7654-139973292300304"]
Jul 29 16:37:08.691844 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:08.691817    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t42tv\" (UniqueName: \"kubernetes.io/projected/35ae8a64-5ef5-4609-9647-38766deab1e0-kube-api-access-t42tv\") pod \"35ae8a64-5ef5-4609-9647-38766deab1e0\" (UID: \"35ae8a64-5ef5-4609-9647-38766deab1e0\") "
Jul 29 16:37:08.693892 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:08.693863    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35ae8a64-5ef5-4609-9647-38766deab1e0-kube-api-access-t42tv" (OuterVolumeSpecName: "kube-api-access-t42tv") pod "35ae8a64-5ef5-4609-9647-38766deab1e0" (UID: "35ae8a64-5ef5-4609-9647-38766deab1e0"). InnerVolumeSpecName "kube-api-access-t42tv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:37:08.793263 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:08.793183    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t42tv\" (UniqueName: \"kubernetes.io/projected/35ae8a64-5ef5-4609-9647-38766deab1e0-kube-api-access-t42tv\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:09.467663 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:09.467635    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a950fedd6d28f616ffa29cbe9f4b68680ce86756271c35e7533d94f22dcb4211"
Jul 29 16:37:09.467838 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:09.467668    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-139973292300304"
Jul 29 16:37:09.478034 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:37:09.478006    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7654-139973292300304\" is forbidden: User \"system:node:ip-10-0-143-118.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-118.ec2.internal' and this object" podUID="35ae8a64-5ef5-4609-9647-38766deab1e0" pod="opendatahub/test-curl-7654-139973292300304"
Jul 29 16:37:10.042870 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:10.042827    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35ae8a64-5ef5-4609-9647-38766deab1e0" path="/var/lib/kubelet/pods/35ae8a64-5ef5-4609-9647-38766deab1e0/volumes"
Jul 29 16:37:10.179447 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:10.179409    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:37:12.486366 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.486331    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf" podUID="c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b" containerName="main" containerID="cri-o://f168623085f75c2154f79a3bbbdcbcae051b1ff8a708207be7d056aaf16bf930" gracePeriod=45
Jul 29 16:37:12.744955 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.744888    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:37:12.823947 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.823903    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-tmp-dir\") pod \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") "
Jul 29 16:37:12.824104 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.823967    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-dshm\") pod \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") "
Jul 29 16:37:12.824104 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.823989    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-tls-certs\") pod \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") "
Jul 29 16:37:12.824104 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.824051    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxb87\" (UniqueName: \"kubernetes.io/projected/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-kube-api-access-wxb87\") pod \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") "
Jul 29 16:37:12.824104 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.824080    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-model-cache\") pod \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") "
Jul 29 16:37:12.824340 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.824129    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-home\") pod \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\" (UID: \"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b\") "
Jul 29 16:37:12.824340 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.824153    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b" (UID: "c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:37:12.824449 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.824344    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-model-cache" (OuterVolumeSpecName: "model-cache") pod "c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b" (UID: "c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:37:12.824449 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.824385    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-home" (OuterVolumeSpecName: "home") pod "c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b" (UID: "c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:37:12.824449 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.824431    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-tmp-dir\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:12.824449 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.824446    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-model-cache\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:12.826100 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.826079    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-kube-api-access-wxb87" (OuterVolumeSpecName: "kube-api-access-wxb87") pod "c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b" (UID: "c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b"). InnerVolumeSpecName "kube-api-access-wxb87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:37:12.826192 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.826108    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-dshm" (OuterVolumeSpecName: "dshm") pod "c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b" (UID: "c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:37:12.826192 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.826117    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b" (UID: "c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:37:12.925252 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.925229    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-home\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:12.925252 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.925250    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-dshm\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:12.925409 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.925259    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-tls-certs\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:12.925409 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:12.925270    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxb87\" (UniqueName: \"kubernetes.io/projected/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b-kube-api-access-wxb87\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:13.415053 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.415015    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt" podUID="981e18f4-ffed-43a9-a86a-123fd6772b64" containerName="main" containerID="cri-o://dc7d7cc6d81ed5f3cb83756d1ccb114b3f42dca5648761009e7288bb2cb5728c" gracePeriod=45
Jul 29 16:37:13.487962 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.487932    2560 generic.go:356] "Generic (PLEG): container finished" podID="c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b" containerID="f168623085f75c2154f79a3bbbdcbcae051b1ff8a708207be7d056aaf16bf930" exitCode=0
Jul 29 16:37:13.488355 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.487984    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"
Jul 29 16:37:13.488355 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.488012    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf" event={"ID":"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b","Type":"ContainerDied","Data":"f168623085f75c2154f79a3bbbdcbcae051b1ff8a708207be7d056aaf16bf930"}
Jul 29 16:37:13.488355 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.488056    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf" event={"ID":"c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b","Type":"ContainerDied","Data":"0a1d5eada0d82998e01703b8743cee9bb6e9636c6201659f319201d97b7cd846"}
Jul 29 16:37:13.488355 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.488074    2560 scope.go:117] "RemoveContainer" containerID="f168623085f75c2154f79a3bbbdcbcae051b1ff8a708207be7d056aaf16bf930"
Jul 29 16:37:13.516698 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.516670    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"]
Jul 29 16:37:13.520700 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.520677    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-81ab8b/rate-test-model-81ab8b-kserve-8489f8b499-fzkcf"]
Jul 29 16:37:13.528037 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.528014    2560 scope.go:117] "RemoveContainer" containerID="f168623085f75c2154f79a3bbbdcbcae051b1ff8a708207be7d056aaf16bf930"
Jul 29 16:37:13.528360 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:37:13.528326    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f168623085f75c2154f79a3bbbdcbcae051b1ff8a708207be7d056aaf16bf930\": container with ID starting with f168623085f75c2154f79a3bbbdcbcae051b1ff8a708207be7d056aaf16bf930 not found: ID does not exist" containerID="f168623085f75c2154f79a3bbbdcbcae051b1ff8a708207be7d056aaf16bf930"
Jul 29 16:37:13.528418 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.528374    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f168623085f75c2154f79a3bbbdcbcae051b1ff8a708207be7d056aaf16bf930"} err="failed to get container status \"f168623085f75c2154f79a3bbbdcbcae051b1ff8a708207be7d056aaf16bf930\": rpc error: code = NotFound desc = could not find container \"f168623085f75c2154f79a3bbbdcbcae051b1ff8a708207be7d056aaf16bf930\": container with ID starting with f168623085f75c2154f79a3bbbdcbcae051b1ff8a708207be7d056aaf16bf930 not found: ID does not exist"
Jul 29 16:37:13.667492 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.667431    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:37:13.732842 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.732812    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/981e18f4-ffed-43a9-a86a-123fd6772b64-tls-certs\") pod \"981e18f4-ffed-43a9-a86a-123fd6772b64\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") "
Jul 29 16:37:13.733018 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.732861    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-home\") pod \"981e18f4-ffed-43a9-a86a-123fd6772b64\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") "
Jul 29 16:37:13.733018 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.732885    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-tmp-dir\") pod \"981e18f4-ffed-43a9-a86a-123fd6772b64\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") "
Jul 29 16:37:13.733018 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.732950    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6s9xl\" (UniqueName: \"kubernetes.io/projected/981e18f4-ffed-43a9-a86a-123fd6772b64-kube-api-access-6s9xl\") pod \"981e18f4-ffed-43a9-a86a-123fd6772b64\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") "
Jul 29 16:37:13.733190 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.733042    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-model-cache\") pod \"981e18f4-ffed-43a9-a86a-123fd6772b64\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") "
Jul 29 16:37:13.733190 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.733068    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-dshm\") pod \"981e18f4-ffed-43a9-a86a-123fd6772b64\" (UID: \"981e18f4-ffed-43a9-a86a-123fd6772b64\") "
Jul 29 16:37:13.733190 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.733164    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-home" (OuterVolumeSpecName: "home") pod "981e18f4-ffed-43a9-a86a-123fd6772b64" (UID: "981e18f4-ffed-43a9-a86a-123fd6772b64"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:37:13.733348 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.733178    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "981e18f4-ffed-43a9-a86a-123fd6772b64" (UID: "981e18f4-ffed-43a9-a86a-123fd6772b64"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:37:13.733348 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.733302    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-model-cache" (OuterVolumeSpecName: "model-cache") pod "981e18f4-ffed-43a9-a86a-123fd6772b64" (UID: "981e18f4-ffed-43a9-a86a-123fd6772b64"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:37:13.733427 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.733385    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-model-cache\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:13.733427 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.733398    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-home\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:13.733427 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.733407    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-tmp-dir\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:13.734998 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.734976    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-dshm" (OuterVolumeSpecName: "dshm") pod "981e18f4-ffed-43a9-a86a-123fd6772b64" (UID: "981e18f4-ffed-43a9-a86a-123fd6772b64"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:37:13.735089 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.735032    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/981e18f4-ffed-43a9-a86a-123fd6772b64-kube-api-access-6s9xl" (OuterVolumeSpecName: "kube-api-access-6s9xl") pod "981e18f4-ffed-43a9-a86a-123fd6772b64" (UID: "981e18f4-ffed-43a9-a86a-123fd6772b64"). InnerVolumeSpecName "kube-api-access-6s9xl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:37:13.735130 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.735097    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/981e18f4-ffed-43a9-a86a-123fd6772b64-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "981e18f4-ffed-43a9-a86a-123fd6772b64" (UID: "981e18f4-ffed-43a9-a86a-123fd6772b64"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:37:13.834025 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.834000    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/981e18f4-ffed-43a9-a86a-123fd6772b64-dshm\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:13.834025 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.834020    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/981e18f4-ffed-43a9-a86a-123fd6772b64-tls-certs\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:13.834168 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:13.834031    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6s9xl\" (UniqueName: \"kubernetes.io/projected/981e18f4-ffed-43a9-a86a-123fd6772b64-kube-api-access-6s9xl\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:14.042851 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.042761    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b" path="/var/lib/kubelet/pods/c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b/volumes"
Jul 29 16:37:14.495507 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.495475    2560 generic.go:356] "Generic (PLEG): container finished" podID="981e18f4-ffed-43a9-a86a-123fd6772b64" containerID="dc7d7cc6d81ed5f3cb83756d1ccb114b3f42dca5648761009e7288bb2cb5728c" exitCode=0
Jul 29 16:37:14.495979 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.495515    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"
Jul 29 16:37:14.495979 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.495557    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt" event={"ID":"981e18f4-ffed-43a9-a86a-123fd6772b64","Type":"ContainerDied","Data":"dc7d7cc6d81ed5f3cb83756d1ccb114b3f42dca5648761009e7288bb2cb5728c"}
Jul 29 16:37:14.495979 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.495600    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt" event={"ID":"981e18f4-ffed-43a9-a86a-123fd6772b64","Type":"ContainerDied","Data":"ec79a58f649bce328b73ed6bfb19d8e49c05f615d0850de6e499dc0ba11e1486"}
Jul 29 16:37:14.495979 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.495620    2560 scope.go:117] "RemoveContainer" containerID="dc7d7cc6d81ed5f3cb83756d1ccb114b3f42dca5648761009e7288bb2cb5728c"
Jul 29 16:37:14.504166 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.504151    2560 scope.go:117] "RemoveContainer" containerID="dc7d7cc6d81ed5f3cb83756d1ccb114b3f42dca5648761009e7288bb2cb5728c"
Jul 29 16:37:14.504405 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:37:14.504387    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc7d7cc6d81ed5f3cb83756d1ccb114b3f42dca5648761009e7288bb2cb5728c\": container with ID starting with dc7d7cc6d81ed5f3cb83756d1ccb114b3f42dca5648761009e7288bb2cb5728c not found: ID does not exist" containerID="dc7d7cc6d81ed5f3cb83756d1ccb114b3f42dca5648761009e7288bb2cb5728c"
Jul 29 16:37:14.504462 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.504413    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc7d7cc6d81ed5f3cb83756d1ccb114b3f42dca5648761009e7288bb2cb5728c"} err="failed to get container status \"dc7d7cc6d81ed5f3cb83756d1ccb114b3f42dca5648761009e7288bb2cb5728c\": rpc error: code = NotFound desc = could not find container \"dc7d7cc6d81ed5f3cb83756d1ccb114b3f42dca5648761009e7288bb2cb5728c\": container with ID starting with dc7d7cc6d81ed5f3cb83756d1ccb114b3f42dca5648761009e7288bb2cb5728c not found: ID does not exist"
Jul 29 16:37:14.512745 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.512708    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"]
Jul 29 16:37:14.514462 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.514442    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-eccd34/rate-test-model-eccd34-kserve-79fcbdf56c-8wtkt"]
Jul 29 16:37:14.779877 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.779804    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7654-139973292301312"]
Jul 29 16:37:14.780191 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.780176    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b" containerName="main"
Jul 29 16:37:14.780241 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.780192    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b" containerName="main"
Jul 29 16:37:14.780241 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.780212    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="981e18f4-ffed-43a9-a86a-123fd6772b64" containerName="main"
Jul 29 16:37:14.780241 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.780221    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="981e18f4-ffed-43a9-a86a-123fd6772b64" containerName="main"
Jul 29 16:37:14.780241 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.780237    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35ae8a64-5ef5-4609-9647-38766deab1e0" containerName="test-curl-7654-139973292300304"
Jul 29 16:37:14.780362 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.780244    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="35ae8a64-5ef5-4609-9647-38766deab1e0" containerName="test-curl-7654-139973292300304"
Jul 29 16:37:14.780362 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.780320    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3f3204c-ef7a-4f07-a0db-e5f4a8b1453b" containerName="main"
Jul 29 16:37:14.780362 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.780330    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="981e18f4-ffed-43a9-a86a-123fd6772b64" containerName="main"
Jul 29 16:37:14.780362 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.780336    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="35ae8a64-5ef5-4609-9647-38766deab1e0" containerName="test-curl-7654-139973292300304"
Jul 29 16:37:14.784519 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.784504    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-139973292301312"
Jul 29 16:37:14.787225 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.787199    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lqpzd\""
Jul 29 16:37:14.790162 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.790139    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7654-139973292301312"]
Jul 29 16:37:14.842782 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.842754    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn95c\" (UniqueName: \"kubernetes.io/projected/013d61ef-b2f6-48b7-b222-d1a939a0c2b5-kube-api-access-nn95c\") pod \"test-curl-7654-139973292301312\" (UID: \"013d61ef-b2f6-48b7-b222-d1a939a0c2b5\") " pod="opendatahub/test-curl-7654-139973292301312"
Jul 29 16:37:14.944046 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.944010    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn95c\" (UniqueName: \"kubernetes.io/projected/013d61ef-b2f6-48b7-b222-d1a939a0c2b5-kube-api-access-nn95c\") pod \"test-curl-7654-139973292301312\" (UID: \"013d61ef-b2f6-48b7-b222-d1a939a0c2b5\") " pod="opendatahub/test-curl-7654-139973292301312"
Jul 29 16:37:14.952647 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:14.952617    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn95c\" (UniqueName: \"kubernetes.io/projected/013d61ef-b2f6-48b7-b222-d1a939a0c2b5-kube-api-access-nn95c\") pod \"test-curl-7654-139973292301312\" (UID: \"013d61ef-b2f6-48b7-b222-d1a939a0c2b5\") " pod="opendatahub/test-curl-7654-139973292301312"
Jul 29 16:37:15.094379 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:15.094290    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-139973292301312"
Jul 29 16:37:15.221221 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:15.221197    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7654-139973292301312"]
Jul 29 16:37:15.223475 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:37:15.223438    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod013d61ef_b2f6_48b7_b222_d1a939a0c2b5.slice/crio-5eecf21f0a77ad241b4cd7f3c6b1bd0ac7bc3e8a0c7d6700a2a14e784879ae38 WatchSource:0}: Error finding container 5eecf21f0a77ad241b4cd7f3c6b1bd0ac7bc3e8a0c7d6700a2a14e784879ae38: Status 404 returned error can't find the container with id 5eecf21f0a77ad241b4cd7f3c6b1bd0ac7bc3e8a0c7d6700a2a14e784879ae38
Jul 29 16:37:15.502323 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:15.502293    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7654-139973292301312" event={"ID":"013d61ef-b2f6-48b7-b222-d1a939a0c2b5","Type":"ContainerStarted","Data":"5eecf21f0a77ad241b4cd7f3c6b1bd0ac7bc3e8a0c7d6700a2a14e784879ae38"}
Jul 29 16:37:16.043884 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:16.043853    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="981e18f4-ffed-43a9-a86a-123fd6772b64" path="/var/lib/kubelet/pods/981e18f4-ffed-43a9-a86a-123fd6772b64/volumes"
Jul 29 16:37:16.509332 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:16.509300    2560 generic.go:356] "Generic (PLEG): container finished" podID="013d61ef-b2f6-48b7-b222-d1a939a0c2b5" containerID="9abce510bc4840d03cf32d88d1207fc6eb3ad88c26405eee267711f6b11f0ff3" exitCode=0
Jul 29 16:37:16.509684 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:16.509386    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7654-139973292301312" event={"ID":"013d61ef-b2f6-48b7-b222-d1a939a0c2b5","Type":"ContainerDied","Data":"9abce510bc4840d03cf32d88d1207fc6eb3ad88c26405eee267711f6b11f0ff3"}
Jul 29 16:37:17.669591 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:17.669568    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-139973292301312"
Jul 29 16:37:17.691410 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:17.691384    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7654-139973292301312_013d61ef-b2f6-48b7-b222-d1a939a0c2b5/test-curl-7654-139973292301312/0.log"
Jul 29 16:37:17.718350 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:17.718318    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7654-139973292301312"]
Jul 29 16:37:17.722216 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:17.722193    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7654-139973292301312"]
Jul 29 16:37:17.769825 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:17.769766    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nn95c\" (UniqueName: \"kubernetes.io/projected/013d61ef-b2f6-48b7-b222-d1a939a0c2b5-kube-api-access-nn95c\") pod \"013d61ef-b2f6-48b7-b222-d1a939a0c2b5\" (UID: \"013d61ef-b2f6-48b7-b222-d1a939a0c2b5\") "
Jul 29 16:37:17.771815 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:17.771792    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/013d61ef-b2f6-48b7-b222-d1a939a0c2b5-kube-api-access-nn95c" (OuterVolumeSpecName: "kube-api-access-nn95c") pod "013d61ef-b2f6-48b7-b222-d1a939a0c2b5" (UID: "013d61ef-b2f6-48b7-b222-d1a939a0c2b5"). InnerVolumeSpecName "kube-api-access-nn95c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:37:17.871044 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:17.871019    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nn95c\" (UniqueName: \"kubernetes.io/projected/013d61ef-b2f6-48b7-b222-d1a939a0c2b5-kube-api-access-nn95c\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:37:18.048977 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:18.048873    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="013d61ef-b2f6-48b7-b222-d1a939a0c2b5" path="/var/lib/kubelet/pods/013d61ef-b2f6-48b7-b222-d1a939a0c2b5/volumes"
Jul 29 16:37:18.520910 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:18.520879    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7654-139973292301312"
Jul 29 16:37:18.520910 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:18.520909    2560 scope.go:117] "RemoveContainer" containerID="9abce510bc4840d03cf32d88d1207fc6eb3ad88c26405eee267711f6b11f0ff3"
Jul 29 16:37:22.732828 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.732790    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"]
Jul 29 16:37:22.733412 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.733372    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="013d61ef-b2f6-48b7-b222-d1a939a0c2b5" containerName="test-curl-7654-139973292301312"
Jul 29 16:37:22.733412 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.733393    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="013d61ef-b2f6-48b7-b222-d1a939a0c2b5" containerName="test-curl-7654-139973292301312"
Jul 29 16:37:22.733541 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.733487    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="013d61ef-b2f6-48b7-b222-d1a939a0c2b5" containerName="test-curl-7654-139973292301312"
Jul 29 16:37:22.737311 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.737290    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"
Jul 29 16:37:22.743383 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.743363    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-777250\""
Jul 29 16:37:22.760284 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.760258    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"]
Jul 29 16:37:22.772737 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.772710    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"]
Jul 29 16:37:22.776767 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.776744    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"
Jul 29 16:37:22.779653 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.779635    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-777250-dockercfg-48mtm\""
Jul 29 16:37:22.787737 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.787716    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"]
Jul 29 16:37:22.811653 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.811623    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgnrw\" (UniqueName: \"kubernetes.io/projected/63bb369b-b9dc-4f63-8c1e-e1728fdefe83-kube-api-access-dgnrw\") pod \"payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8\" (UID: \"63bb369b-b9dc-4f63-8c1e-e1728fdefe83\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"
Jul 29 16:37:22.811791 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.811665    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63bb369b-b9dc-4f63-8c1e-e1728fdefe83-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8\" (UID: \"63bb369b-b9dc-4f63-8c1e-e1728fdefe83\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"
Jul 29 16:37:22.811791 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.811684    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8h49\" (UniqueName: \"kubernetes.io/projected/dc3573cc-cbb7-44eb-8192-aa8550053190-kube-api-access-q8h49\") pod \"payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz\" (UID: \"dc3573cc-cbb7-44eb-8192-aa8550053190\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"
Jul 29 16:37:22.811791 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.811783    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc3573cc-cbb7-44eb-8192-aa8550053190-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz\" (UID: \"dc3573cc-cbb7-44eb-8192-aa8550053190\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"
Jul 29 16:37:22.912234 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.912199    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgnrw\" (UniqueName: \"kubernetes.io/projected/63bb369b-b9dc-4f63-8c1e-e1728fdefe83-kube-api-access-dgnrw\") pod \"payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8\" (UID: \"63bb369b-b9dc-4f63-8c1e-e1728fdefe83\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"
Jul 29 16:37:22.912234 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.912240    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63bb369b-b9dc-4f63-8c1e-e1728fdefe83-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8\" (UID: \"63bb369b-b9dc-4f63-8c1e-e1728fdefe83\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"
Jul 29 16:37:22.912424 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.912256    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8h49\" (UniqueName: \"kubernetes.io/projected/dc3573cc-cbb7-44eb-8192-aa8550053190-kube-api-access-q8h49\") pod \"payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz\" (UID: \"dc3573cc-cbb7-44eb-8192-aa8550053190\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"
Jul 29 16:37:22.912474 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.912449    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc3573cc-cbb7-44eb-8192-aa8550053190-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz\" (UID: \"dc3573cc-cbb7-44eb-8192-aa8550053190\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"
Jul 29 16:37:22.913002 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.912978    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63bb369b-b9dc-4f63-8c1e-e1728fdefe83-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8\" (UID: \"63bb369b-b9dc-4f63-8c1e-e1728fdefe83\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"
Jul 29 16:37:22.913227 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.913209    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc3573cc-cbb7-44eb-8192-aa8550053190-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz\" (UID: \"dc3573cc-cbb7-44eb-8192-aa8550053190\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"
Jul 29 16:37:22.921530 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.921509    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8h49\" (UniqueName: \"kubernetes.io/projected/dc3573cc-cbb7-44eb-8192-aa8550053190-kube-api-access-q8h49\") pod \"payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz\" (UID: \"dc3573cc-cbb7-44eb-8192-aa8550053190\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"
Jul 29 16:37:22.921609 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:22.921536    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgnrw\" (UniqueName: \"kubernetes.io/projected/63bb369b-b9dc-4f63-8c1e-e1728fdefe83-kube-api-access-dgnrw\") pod \"payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8\" (UID: \"63bb369b-b9dc-4f63-8c1e-e1728fdefe83\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"
Jul 29 16:37:23.048024 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:23.047940    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"
Jul 29 16:37:23.086597 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:23.086563    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"
Jul 29 16:37:23.200071 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:23.200045    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"]
Jul 29 16:37:23.202049 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:37:23.201996    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc3573cc_cbb7_44eb_8192_aa8550053190.slice/crio-1d26e5f163e553f5503a4e62ae74f428dcbf09636253567beee7c467bdc52772 WatchSource:0}: Error finding container 1d26e5f163e553f5503a4e62ae74f428dcbf09636253567beee7c467bdc52772: Status 404 returned error can't find the container with id 1d26e5f163e553f5503a4e62ae74f428dcbf09636253567beee7c467bdc52772
Jul 29 16:37:23.441752 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:23.441723    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"]
Jul 29 16:37:23.443552 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:37:23.443526    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63bb369b_b9dc_4f63_8c1e_e1728fdefe83.slice/crio-0efb8d2c5882136992d4e8845df37d1c80be732c576bd9fb5018416733d1d58d WatchSource:0}: Error finding container 0efb8d2c5882136992d4e8845df37d1c80be732c576bd9fb5018416733d1d58d: Status 404 returned error can't find the container with id 0efb8d2c5882136992d4e8845df37d1c80be732c576bd9fb5018416733d1d58d
Jul 29 16:37:23.548391 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:23.548354    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8" event={"ID":"63bb369b-b9dc-4f63-8c1e-e1728fdefe83","Type":"ContainerStarted","Data":"69cc1697f98885252fb3a150569ac5268d5b08dff39b18adedc7ddc993363bed"}
Jul 29 16:37:23.548556 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:23.548410    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8" event={"ID":"63bb369b-b9dc-4f63-8c1e-e1728fdefe83","Type":"ContainerStarted","Data":"0efb8d2c5882136992d4e8845df37d1c80be732c576bd9fb5018416733d1d58d"}
Jul 29 16:37:23.548556 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:23.548533    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"
Jul 29 16:37:23.550155 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:23.550127    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz" event={"ID":"dc3573cc-cbb7-44eb-8192-aa8550053190","Type":"ContainerStarted","Data":"c3bcf1ae7587da60012362e1f3bf81f430f4a44608e73345877845518d9972f5"}
Jul 29 16:37:23.550289 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:23.550170    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz" event={"ID":"dc3573cc-cbb7-44eb-8192-aa8550053190","Type":"ContainerStarted","Data":"1d26e5f163e553f5503a4e62ae74f428dcbf09636253567beee7c467bdc52772"}
Jul 29 16:37:23.550289 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:23.550192    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"
Jul 29 16:37:23.568490 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:23.568446    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8" podStartSLOduration=1.568433267 podStartE2EDuration="1.568433267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:37:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:37:23.566767511 +0000 UTC m=+3048.008973770" watchObservedRunningTime="2026-07-29 16:37:23.568433267 +0000 UTC m=+3048.010639526"
Jul 29 16:37:23.586895 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:23.586858    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz" podStartSLOduration=1.586844973 podStartE2EDuration="1.586844973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:37:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:37:23.585927062 +0000 UTC m=+3048.028133311" watchObservedRunningTime="2026-07-29 16:37:23.586844973 +0000 UTC m=+3048.029051223"
Jul 29 16:37:27.252896 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:27.252861    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:37:34.557555 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:34.557523    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"
Jul 29 16:37:34.557995 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:34.557603    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"
Jul 29 16:37:51.622947 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.622900    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"]
Jul 29 16:37:51.626595 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.626578    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"
Jul 29 16:37:51.630437 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.630418    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-3c2006\""
Jul 29 16:37:51.631035 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.631017    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-3c2006-dockercfg-zds9j\""
Jul 29 16:37:51.642077 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.642055    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"]
Jul 29 16:37:51.665576 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.665550    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"]
Jul 29 16:37:51.668884 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.668867    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"
Jul 29 16:37:51.678523 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.678503    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"]
Jul 29 16:37:51.770273 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.770221    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3e12246-d240-45ef-8861-19b68ac5f7e4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m\" (UID: \"e3e12246-d240-45ef-8861-19b68ac5f7e4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"
Jul 29 16:37:51.770273 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.770272    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/625103f6-4a40-4977-9276-a920ed0640b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8\" (UID: \"625103f6-4a40-4977-9276-a920ed0640b4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"
Jul 29 16:37:51.770501 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.770290    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvphg\" (UniqueName: \"kubernetes.io/projected/625103f6-4a40-4977-9276-a920ed0640b4-kube-api-access-pvphg\") pod \"payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8\" (UID: \"625103f6-4a40-4977-9276-a920ed0640b4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"
Jul 29 16:37:51.770501 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.770386    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4zh9\" (UniqueName: \"kubernetes.io/projected/e3e12246-d240-45ef-8861-19b68ac5f7e4-kube-api-access-p4zh9\") pod \"payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m\" (UID: \"e3e12246-d240-45ef-8861-19b68ac5f7e4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"
Jul 29 16:37:51.871139 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.871102    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4zh9\" (UniqueName: \"kubernetes.io/projected/e3e12246-d240-45ef-8861-19b68ac5f7e4-kube-api-access-p4zh9\") pod \"payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m\" (UID: \"e3e12246-d240-45ef-8861-19b68ac5f7e4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"
Jul 29 16:37:51.871309 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.871181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3e12246-d240-45ef-8861-19b68ac5f7e4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m\" (UID: \"e3e12246-d240-45ef-8861-19b68ac5f7e4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"
Jul 29 16:37:51.871309 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.871203    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/625103f6-4a40-4977-9276-a920ed0640b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8\" (UID: \"625103f6-4a40-4977-9276-a920ed0640b4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"
Jul 29 16:37:51.871309 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.871219    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvphg\" (UniqueName: \"kubernetes.io/projected/625103f6-4a40-4977-9276-a920ed0640b4-kube-api-access-pvphg\") pod \"payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8\" (UID: \"625103f6-4a40-4977-9276-a920ed0640b4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"
Jul 29 16:37:51.871966 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.871947    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/625103f6-4a40-4977-9276-a920ed0640b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8\" (UID: \"625103f6-4a40-4977-9276-a920ed0640b4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"
Jul 29 16:37:51.872004 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.871963    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3e12246-d240-45ef-8861-19b68ac5f7e4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m\" (UID: \"e3e12246-d240-45ef-8861-19b68ac5f7e4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"
Jul 29 16:37:51.879728 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.879663    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvphg\" (UniqueName: \"kubernetes.io/projected/625103f6-4a40-4977-9276-a920ed0640b4-kube-api-access-pvphg\") pod \"payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8\" (UID: \"625103f6-4a40-4977-9276-a920ed0640b4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"
Jul 29 16:37:51.879843 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.879775    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4zh9\" (UniqueName: \"kubernetes.io/projected/e3e12246-d240-45ef-8861-19b68ac5f7e4-kube-api-access-p4zh9\") pod \"payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m\" (UID: \"e3e12246-d240-45ef-8861-19b68ac5f7e4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"
Jul 29 16:37:51.936419 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.936389    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"
Jul 29 16:37:51.979380 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:51.979350    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"
Jul 29 16:37:52.104962 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:52.104908    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"]
Jul 29 16:37:52.106910 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:37:52.106880    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e12246_d240_45ef_8861_19b68ac5f7e4.slice/crio-f39069572fd738f309099d373c4dc49ef3a6dc1ce23daf9a232247e48b34875c WatchSource:0}: Error finding container f39069572fd738f309099d373c4dc49ef3a6dc1ce23daf9a232247e48b34875c: Status 404 returned error can't find the container with id f39069572fd738f309099d373c4dc49ef3a6dc1ce23daf9a232247e48b34875c
Jul 29 16:37:52.151254 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:52.151231    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"]
Jul 29 16:37:52.152764 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:37:52.152741    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod625103f6_4a40_4977_9276_a920ed0640b4.slice/crio-102da7f32d8f4a1e5b81c896eb19a8c38bcccf7f83a22d65432e54184056aaaf WatchSource:0}: Error finding container 102da7f32d8f4a1e5b81c896eb19a8c38bcccf7f83a22d65432e54184056aaaf: Status 404 returned error can't find the container with id 102da7f32d8f4a1e5b81c896eb19a8c38bcccf7f83a22d65432e54184056aaaf
Jul 29 16:37:52.697268 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:52.697231    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m" event={"ID":"e3e12246-d240-45ef-8861-19b68ac5f7e4","Type":"ContainerStarted","Data":"f04670750b4216dea2f745f81a5cf9b2e271c046bf33595012232f2c6881f762"}
Jul 29 16:37:52.697697 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:52.697304    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"
Jul 29 16:37:52.697697 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:52.697331    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m" event={"ID":"e3e12246-d240-45ef-8861-19b68ac5f7e4","Type":"ContainerStarted","Data":"f39069572fd738f309099d373c4dc49ef3a6dc1ce23daf9a232247e48b34875c"}
Jul 29 16:37:52.699119 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:52.699093    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8" event={"ID":"625103f6-4a40-4977-9276-a920ed0640b4","Type":"ContainerStarted","Data":"8bd659d2b66a3979e20c10d635c4198ce803d9f05246f414eef8fac741246394"}
Jul 29 16:37:52.699256 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:52.699139    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8" event={"ID":"625103f6-4a40-4977-9276-a920ed0640b4","Type":"ContainerStarted","Data":"102da7f32d8f4a1e5b81c896eb19a8c38bcccf7f83a22d65432e54184056aaaf"}
Jul 29 16:37:52.699256 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:52.699182    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"
Jul 29 16:37:52.717232 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:52.717185    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m" podStartSLOduration=1.717167495 podStartE2EDuration="1.717167495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:37:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:37:52.715379182 +0000 UTC m=+3077.157585465" watchObservedRunningTime="2026-07-29 16:37:52.717167495 +0000 UTC m=+3077.159373753"
Jul 29 16:37:52.736897 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:37:52.736850    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8" podStartSLOduration=1.7368365940000001 podStartE2EDuration="1.736836594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:37:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:37:52.734883087 +0000 UTC m=+3077.177089347" watchObservedRunningTime="2026-07-29 16:37:52.736836594 +0000 UTC m=+3077.179042857"
Jul 29 16:38:00.456866 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:00.456822    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:38:03.709639 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:03.709610    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"
Jul 29 16:38:03.710065 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:03.709856    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"
Jul 29 16:38:27.852352 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:27.852324    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m_e3e12246-d240-45ef-8861-19b68ac5f7e4/payload-processing/0.log"
Jul 29 16:38:33.536252 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.536218    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"]
Jul 29 16:38:33.540231 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.540208    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.544168 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.544146    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-777250\"/\"default-dockercfg-vltwf\""
Jul 29 16:38:33.544304 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.544274    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-777250\"/\"kube-root-ca.crt\""
Jul 29 16:38:33.544304 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.544293    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-777250\"/\"openshift-service-ca.crt\""
Jul 29 16:38:33.544434 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.544159    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-777250\"/\"ipp-route-777250-kserve-self-signed-certs\""
Jul 29 16:38:33.550198 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.550172    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"]
Jul 29 16:38:33.638397 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.638363    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-tmp-dir\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.638397 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.638399    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-dshm\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.638589 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.638443    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9493df19-2f9e-4514-8cdc-99431b5a0cce-tls-certs\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.638589 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.638461    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7p5l\" (UniqueName: \"kubernetes.io/projected/9493df19-2f9e-4514-8cdc-99431b5a0cce-kube-api-access-d7p5l\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.638589 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.638480    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-home\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.638589 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.638556    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-model-cache\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.739063 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.739029    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-model-cache\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.739215 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.739090    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-tmp-dir\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.739215 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.739108    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-dshm\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.739215 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.739143    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9493df19-2f9e-4514-8cdc-99431b5a0cce-tls-certs\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.739215 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.739159    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7p5l\" (UniqueName: \"kubernetes.io/projected/9493df19-2f9e-4514-8cdc-99431b5a0cce-kube-api-access-d7p5l\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.739215 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.739179    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-home\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.739469 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.739423    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-model-cache\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.739543 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.739523    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-home\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.739609 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.739587    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-tmp-dir\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.741345 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.741316    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-dshm\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.741526 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.741510    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9493df19-2f9e-4514-8cdc-99431b5a0cce-tls-certs\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.747172 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.747149    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7p5l\" (UniqueName: \"kubernetes.io/projected/9493df19-2f9e-4514-8cdc-99431b5a0cce-kube-api-access-d7p5l\") pod \"ipp-route-777250-kserve-64489f9659-5qqmx\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") " pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.850850 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.850780    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:33.980822 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:33.980799    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"]
Jul 29 16:38:33.982931 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:38:33.982891    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9493df19_2f9e_4514_8cdc_99431b5a0cce.slice/crio-be1322ea80b77d223f10f89545dd34c73dcac174f10208cc8b643c0457fb37bb WatchSource:0}: Error finding container be1322ea80b77d223f10f89545dd34c73dcac174f10208cc8b643c0457fb37bb: Status 404 returned error can't find the container with id be1322ea80b77d223f10f89545dd34c73dcac174f10208cc8b643c0457fb37bb
Jul 29 16:38:34.917992 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:34.917945    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx" event={"ID":"9493df19-2f9e-4514-8cdc-99431b5a0cce","Type":"ContainerStarted","Data":"714c64c2ad156c2b177b06543554f772ab5613b2f622facf204b97b9090b825f"}
Jul 29 16:38:34.918464 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:34.918003    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx" event={"ID":"9493df19-2f9e-4514-8cdc-99431b5a0cce","Type":"ContainerStarted","Data":"be1322ea80b77d223f10f89545dd34c73dcac174f10208cc8b643c0457fb37bb"}
Jul 29 16:38:34.936144 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:34.936094    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx" podStartSLOduration=1.936077063 podStartE2EDuration="1.936077063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:38:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:38:34.934492482 +0000 UTC m=+3119.376698742" watchObservedRunningTime="2026-07-29 16:38:34.936077063 +0000 UTC m=+3119.378283323"
Jul 29 16:38:37.555143 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:37.555109    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:38:43.851602 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:43.851571    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:43.852196 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:43.851661    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:43.864255 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:43.864228    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:38:43.982894 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:38:43.982868    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:39:03.154727 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:03.154644    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:39:14.826818 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:14.826792    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8_63bb369b-b9dc-4f63-8c1e-e1728fdefe83/payload-processing/0.log"
Jul 29 16:39:15.071482 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:15.071457    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m_e3e12246-d240-45ef-8861-19b68ac5f7e4/payload-processing/0.log"
Jul 29 16:39:20.056489 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:20.056455    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"]
Jul 29 16:39:20.059851 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:20.059835    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"
Jul 29 16:39:20.062790 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:20.062767    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-1bad22\""
Jul 29 16:39:20.062790 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:20.062788    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-1bad22-dockercfg-pmm4h\""
Jul 29 16:39:20.068484 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:20.068458    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"]
Jul 29 16:39:20.138552 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:20.138517    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccw5j\" (UniqueName: \"kubernetes.io/projected/c56b9835-dc19-4e16-a04d-83faa1afdec9-kube-api-access-ccw5j\") pod \"payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx\" (UID: \"c56b9835-dc19-4e16-a04d-83faa1afdec9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"
Jul 29 16:39:20.138718 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:20.138578    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56b9835-dc19-4e16-a04d-83faa1afdec9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx\" (UID: \"c56b9835-dc19-4e16-a04d-83faa1afdec9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"
Jul 29 16:39:20.239731 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:20.239701    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56b9835-dc19-4e16-a04d-83faa1afdec9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx\" (UID: \"c56b9835-dc19-4e16-a04d-83faa1afdec9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"
Jul 29 16:39:20.239907 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:20.239805    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccw5j\" (UniqueName: \"kubernetes.io/projected/c56b9835-dc19-4e16-a04d-83faa1afdec9-kube-api-access-ccw5j\") pod \"payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx\" (UID: \"c56b9835-dc19-4e16-a04d-83faa1afdec9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"
Jul 29 16:39:20.240435 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:20.240414    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56b9835-dc19-4e16-a04d-83faa1afdec9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx\" (UID: \"c56b9835-dc19-4e16-a04d-83faa1afdec9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"
Jul 29 16:39:20.254020 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:20.253997    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccw5j\" (UniqueName: \"kubernetes.io/projected/c56b9835-dc19-4e16-a04d-83faa1afdec9-kube-api-access-ccw5j\") pod \"payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx\" (UID: \"c56b9835-dc19-4e16-a04d-83faa1afdec9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"
Jul 29 16:39:20.371209 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:20.371135    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"
Jul 29 16:39:20.712807 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:20.712781    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"]
Jul 29 16:39:20.714708 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:39:20.714680    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc56b9835_dc19_4e16_a04d_83faa1afdec9.slice/crio-1e9cbe2803bfcc23adaa104e1af71e1cee7e4e572ae7534abf0d2641f8cd299d WatchSource:0}: Error finding container 1e9cbe2803bfcc23adaa104e1af71e1cee7e4e572ae7534abf0d2641f8cd299d: Status 404 returned error can't find the container with id 1e9cbe2803bfcc23adaa104e1af71e1cee7e4e572ae7534abf0d2641f8cd299d
Jul 29 16:39:21.176348 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:21.176313    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx" event={"ID":"c56b9835-dc19-4e16-a04d-83faa1afdec9","Type":"ContainerStarted","Data":"8df1a96675082036bc4ce431cb429c2ebf92cb4f19c50fc927aa7812d502ad0d"}
Jul 29 16:39:21.176783 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:21.176381    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"
Jul 29 16:39:21.176783 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:21.176411    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx" event={"ID":"c56b9835-dc19-4e16-a04d-83faa1afdec9","Type":"ContainerStarted","Data":"1e9cbe2803bfcc23adaa104e1af71e1cee7e4e572ae7534abf0d2641f8cd299d"}
Jul 29 16:39:21.192852 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:21.192805    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx" podStartSLOduration=1.192791654 podStartE2EDuration="1.192791654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:39:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:39:21.19178711 +0000 UTC m=+3165.633993394" watchObservedRunningTime="2026-07-29 16:39:21.192791654 +0000 UTC m=+3165.634997914"
Jul 29 16:39:23.758346 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:23.758311    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:39:32.184522 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:32.184489    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"
Jul 29 16:39:49.904101 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:49.904067    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"]
Jul 29 16:39:49.904638 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:49.904348    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx" podUID="c56b9835-dc19-4e16-a04d-83faa1afdec9" containerName="payload-pre-processing" containerID="cri-o://8df1a96675082036bc4ce431cb429c2ebf92cb4f19c50fc927aa7812d502ad0d" gracePeriod=30
Jul 29 16:39:50.164885 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.164827    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"
Jul 29 16:39:50.203049 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.203020    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56b9835-dc19-4e16-a04d-83faa1afdec9-plugins-config-volume\") pod \"c56b9835-dc19-4e16-a04d-83faa1afdec9\" (UID: \"c56b9835-dc19-4e16-a04d-83faa1afdec9\") "
Jul 29 16:39:50.203202 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.203058    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccw5j\" (UniqueName: \"kubernetes.io/projected/c56b9835-dc19-4e16-a04d-83faa1afdec9-kube-api-access-ccw5j\") pod \"c56b9835-dc19-4e16-a04d-83faa1afdec9\" (UID: \"c56b9835-dc19-4e16-a04d-83faa1afdec9\") "
Jul 29 16:39:50.203373 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.203350    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c56b9835-dc19-4e16-a04d-83faa1afdec9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c56b9835-dc19-4e16-a04d-83faa1afdec9" (UID: "c56b9835-dc19-4e16-a04d-83faa1afdec9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:39:50.205213 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.205182    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c56b9835-dc19-4e16-a04d-83faa1afdec9-kube-api-access-ccw5j" (OuterVolumeSpecName: "kube-api-access-ccw5j") pod "c56b9835-dc19-4e16-a04d-83faa1afdec9" (UID: "c56b9835-dc19-4e16-a04d-83faa1afdec9"). InnerVolumeSpecName "kube-api-access-ccw5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:39:50.303934 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.303868    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56b9835-dc19-4e16-a04d-83faa1afdec9-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:39:50.303934 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.303908    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccw5j\" (UniqueName: \"kubernetes.io/projected/c56b9835-dc19-4e16-a04d-83faa1afdec9-kube-api-access-ccw5j\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:39:50.330661 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.330621    2560 generic.go:356] "Generic (PLEG): container finished" podID="c56b9835-dc19-4e16-a04d-83faa1afdec9" containerID="8df1a96675082036bc4ce431cb429c2ebf92cb4f19c50fc927aa7812d502ad0d" exitCode=0
Jul 29 16:39:50.330824 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.330669    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"
Jul 29 16:39:50.330824 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.330702    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx" event={"ID":"c56b9835-dc19-4e16-a04d-83faa1afdec9","Type":"ContainerDied","Data":"8df1a96675082036bc4ce431cb429c2ebf92cb4f19c50fc927aa7812d502ad0d"}
Jul 29 16:39:50.330824 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.330757    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx" event={"ID":"c56b9835-dc19-4e16-a04d-83faa1afdec9","Type":"ContainerDied","Data":"1e9cbe2803bfcc23adaa104e1af71e1cee7e4e572ae7534abf0d2641f8cd299d"}
Jul 29 16:39:50.330824 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.330784    2560 scope.go:117] "RemoveContainer" containerID="8df1a96675082036bc4ce431cb429c2ebf92cb4f19c50fc927aa7812d502ad0d"
Jul 29 16:39:50.339946 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.339902    2560 scope.go:117] "RemoveContainer" containerID="8df1a96675082036bc4ce431cb429c2ebf92cb4f19c50fc927aa7812d502ad0d"
Jul 29 16:39:50.340187 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:39:50.340167    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8df1a96675082036bc4ce431cb429c2ebf92cb4f19c50fc927aa7812d502ad0d\": container with ID starting with 8df1a96675082036bc4ce431cb429c2ebf92cb4f19c50fc927aa7812d502ad0d not found: ID does not exist" containerID="8df1a96675082036bc4ce431cb429c2ebf92cb4f19c50fc927aa7812d502ad0d"
Jul 29 16:39:50.340245 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.340195    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8df1a96675082036bc4ce431cb429c2ebf92cb4f19c50fc927aa7812d502ad0d"} err="failed to get container status \"8df1a96675082036bc4ce431cb429c2ebf92cb4f19c50fc927aa7812d502ad0d\": rpc error: code = NotFound desc = could not find container \"8df1a96675082036bc4ce431cb429c2ebf92cb4f19c50fc927aa7812d502ad0d\": container with ID starting with 8df1a96675082036bc4ce431cb429c2ebf92cb4f19c50fc927aa7812d502ad0d not found: ID does not exist"
Jul 29 16:39:50.352407 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.352385    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"]
Jul 29 16:39:50.357616 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:50.357589    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1bad22-759c6c6844-wzcdx"]
Jul 29 16:39:52.043076 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:52.043044    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c56b9835-dc19-4e16-a04d-83faa1afdec9" path="/var/lib/kubelet/pods/c56b9835-dc19-4e16-a04d-83faa1afdec9/volumes"
Jul 29 16:39:54.258530 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:39:54.258498    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:40:15.751105 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:15.751069    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"]
Jul 29 16:40:15.751574 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:15.751331    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8" podUID="63bb369b-b9dc-4f63-8c1e-e1728fdefe83" containerName="payload-processing" containerID="cri-o://69cc1697f98885252fb3a150569ac5268d5b08dff39b18adedc7ddc993363bed" gracePeriod=30
Jul 29 16:40:15.759812 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:15.759783    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"]
Jul 29 16:40:15.760010 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:15.759987    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz" podUID="dc3573cc-cbb7-44eb-8192-aa8550053190" containerName="payload-pre-processing" containerID="cri-o://c3bcf1ae7587da60012362e1f3bf81f430f4a44608e73345877845518d9972f5" gracePeriod=30
Jul 29 16:40:16.016744 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.016681    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"
Jul 29 16:40:16.047995 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.047972    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"
Jul 29 16:40:16.137096 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.137060    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63bb369b-b9dc-4f63-8c1e-e1728fdefe83-plugins-config-volume\") pod \"63bb369b-b9dc-4f63-8c1e-e1728fdefe83\" (UID: \"63bb369b-b9dc-4f63-8c1e-e1728fdefe83\") "
Jul 29 16:40:16.137271 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.137117    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc3573cc-cbb7-44eb-8192-aa8550053190-plugins-config-volume\") pod \"dc3573cc-cbb7-44eb-8192-aa8550053190\" (UID: \"dc3573cc-cbb7-44eb-8192-aa8550053190\") "
Jul 29 16:40:16.137271 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.137179    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgnrw\" (UniqueName: \"kubernetes.io/projected/63bb369b-b9dc-4f63-8c1e-e1728fdefe83-kube-api-access-dgnrw\") pod \"63bb369b-b9dc-4f63-8c1e-e1728fdefe83\" (UID: \"63bb369b-b9dc-4f63-8c1e-e1728fdefe83\") "
Jul 29 16:40:16.137271 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.137195    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8h49\" (UniqueName: \"kubernetes.io/projected/dc3573cc-cbb7-44eb-8192-aa8550053190-kube-api-access-q8h49\") pod \"dc3573cc-cbb7-44eb-8192-aa8550053190\" (UID: \"dc3573cc-cbb7-44eb-8192-aa8550053190\") "
Jul 29 16:40:16.137499 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.137475    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63bb369b-b9dc-4f63-8c1e-e1728fdefe83-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "63bb369b-b9dc-4f63-8c1e-e1728fdefe83" (UID: "63bb369b-b9dc-4f63-8c1e-e1728fdefe83"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:40:16.137552 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.137479    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc3573cc-cbb7-44eb-8192-aa8550053190-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc3573cc-cbb7-44eb-8192-aa8550053190" (UID: "dc3573cc-cbb7-44eb-8192-aa8550053190"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:40:16.139289 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.139268    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc3573cc-cbb7-44eb-8192-aa8550053190-kube-api-access-q8h49" (OuterVolumeSpecName: "kube-api-access-q8h49") pod "dc3573cc-cbb7-44eb-8192-aa8550053190" (UID: "dc3573cc-cbb7-44eb-8192-aa8550053190"). InnerVolumeSpecName "kube-api-access-q8h49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:40:16.139348 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.139309    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63bb369b-b9dc-4f63-8c1e-e1728fdefe83-kube-api-access-dgnrw" (OuterVolumeSpecName: "kube-api-access-dgnrw") pod "63bb369b-b9dc-4f63-8c1e-e1728fdefe83" (UID: "63bb369b-b9dc-4f63-8c1e-e1728fdefe83"). InnerVolumeSpecName "kube-api-access-dgnrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:40:16.238807 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.238776    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63bb369b-b9dc-4f63-8c1e-e1728fdefe83-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:40:16.238807 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.238801    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc3573cc-cbb7-44eb-8192-aa8550053190-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:40:16.238807 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.238813    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgnrw\" (UniqueName: \"kubernetes.io/projected/63bb369b-b9dc-4f63-8c1e-e1728fdefe83-kube-api-access-dgnrw\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:40:16.239055 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.238823    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8h49\" (UniqueName: \"kubernetes.io/projected/dc3573cc-cbb7-44eb-8192-aa8550053190-kube-api-access-q8h49\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:40:16.252611 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.252582    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:40:16.466438 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.466401    2560 generic.go:356] "Generic (PLEG): container finished" podID="63bb369b-b9dc-4f63-8c1e-e1728fdefe83" containerID="69cc1697f98885252fb3a150569ac5268d5b08dff39b18adedc7ddc993363bed" exitCode=0
Jul 29 16:40:16.466619 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.466445    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"
Jul 29 16:40:16.466619 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.466491    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8" event={"ID":"63bb369b-b9dc-4f63-8c1e-e1728fdefe83","Type":"ContainerDied","Data":"69cc1697f98885252fb3a150569ac5268d5b08dff39b18adedc7ddc993363bed"}
Jul 29 16:40:16.466619 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.466552    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8" event={"ID":"63bb369b-b9dc-4f63-8c1e-e1728fdefe83","Type":"ContainerDied","Data":"0efb8d2c5882136992d4e8845df37d1c80be732c576bd9fb5018416733d1d58d"}
Jul 29 16:40:16.466619 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.466585    2560 scope.go:117] "RemoveContainer" containerID="69cc1697f98885252fb3a150569ac5268d5b08dff39b18adedc7ddc993363bed"
Jul 29 16:40:16.467946 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.467902    2560 generic.go:356] "Generic (PLEG): container finished" podID="dc3573cc-cbb7-44eb-8192-aa8550053190" containerID="c3bcf1ae7587da60012362e1f3bf81f430f4a44608e73345877845518d9972f5" exitCode=0
Jul 29 16:40:16.468054 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.467948    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz" event={"ID":"dc3573cc-cbb7-44eb-8192-aa8550053190","Type":"ContainerDied","Data":"c3bcf1ae7587da60012362e1f3bf81f430f4a44608e73345877845518d9972f5"}
Jul 29 16:40:16.468054 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.467979    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"
Jul 29 16:40:16.468054 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.467996    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz" event={"ID":"dc3573cc-cbb7-44eb-8192-aa8550053190","Type":"ContainerDied","Data":"1d26e5f163e553f5503a4e62ae74f428dcbf09636253567beee7c467bdc52772"}
Jul 29 16:40:16.475958 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.475936    2560 scope.go:117] "RemoveContainer" containerID="69cc1697f98885252fb3a150569ac5268d5b08dff39b18adedc7ddc993363bed"
Jul 29 16:40:16.476245 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:40:16.476222    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69cc1697f98885252fb3a150569ac5268d5b08dff39b18adedc7ddc993363bed\": container with ID starting with 69cc1697f98885252fb3a150569ac5268d5b08dff39b18adedc7ddc993363bed not found: ID does not exist" containerID="69cc1697f98885252fb3a150569ac5268d5b08dff39b18adedc7ddc993363bed"
Jul 29 16:40:16.476336 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.476260    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69cc1697f98885252fb3a150569ac5268d5b08dff39b18adedc7ddc993363bed"} err="failed to get container status \"69cc1697f98885252fb3a150569ac5268d5b08dff39b18adedc7ddc993363bed\": rpc error: code = NotFound desc = could not find container \"69cc1697f98885252fb3a150569ac5268d5b08dff39b18adedc7ddc993363bed\": container with ID starting with 69cc1697f98885252fb3a150569ac5268d5b08dff39b18adedc7ddc993363bed not found: ID does not exist"
Jul 29 16:40:16.476336 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.476285    2560 scope.go:117] "RemoveContainer" containerID="c3bcf1ae7587da60012362e1f3bf81f430f4a44608e73345877845518d9972f5"
Jul 29 16:40:16.484669 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.484654    2560 scope.go:117] "RemoveContainer" containerID="c3bcf1ae7587da60012362e1f3bf81f430f4a44608e73345877845518d9972f5"
Jul 29 16:40:16.484898 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:40:16.484879    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3bcf1ae7587da60012362e1f3bf81f430f4a44608e73345877845518d9972f5\": container with ID starting with c3bcf1ae7587da60012362e1f3bf81f430f4a44608e73345877845518d9972f5 not found: ID does not exist" containerID="c3bcf1ae7587da60012362e1f3bf81f430f4a44608e73345877845518d9972f5"
Jul 29 16:40:16.485031 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.484909    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3bcf1ae7587da60012362e1f3bf81f430f4a44608e73345877845518d9972f5"} err="failed to get container status \"c3bcf1ae7587da60012362e1f3bf81f430f4a44608e73345877845518d9972f5\": rpc error: code = NotFound desc = could not find container \"c3bcf1ae7587da60012362e1f3bf81f430f4a44608e73345877845518d9972f5\": container with ID starting with c3bcf1ae7587da60012362e1f3bf81f430f4a44608e73345877845518d9972f5 not found: ID does not exist"
Jul 29 16:40:16.505564 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.505541    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"]
Jul 29 16:40:16.515890 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.515867    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-777250-58574d7cb4-l55kz"]
Jul 29 16:40:16.533531 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.533510    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"]
Jul 29 16:40:16.542192 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:16.542171    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-777250-7c457659f8-2zsh8"]
Jul 29 16:40:18.045520 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:18.045483    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63bb369b-b9dc-4f63-8c1e-e1728fdefe83" path="/var/lib/kubelet/pods/63bb369b-b9dc-4f63-8c1e-e1728fdefe83/volumes"
Jul 29 16:40:18.046053 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:18.046031    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc3573cc-cbb7-44eb-8192-aa8550053190" path="/var/lib/kubelet/pods/dc3573cc-cbb7-44eb-8192-aa8550053190/volumes"
Jul 29 16:40:21.270207 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:21.270169    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"]
Jul 29 16:40:27.553379 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:27.553344    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:40:36.288991 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.288935    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx" podUID="9493df19-2f9e-4514-8cdc-99431b5a0cce" containerName="main" containerID="cri-o://714c64c2ad156c2b177b06543554f772ab5613b2f622facf204b97b9090b825f" gracePeriod=45
Jul 29 16:40:36.365878 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.365844    2560 scope.go:117] "RemoveContainer" containerID="1649af6b43b5c8638c321e5799dc4d9a14f7587ea5eb166294353763bc78bb6f"
Jul 29 16:40:36.548524 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.548466    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:40:36.582140 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.582104    2560 generic.go:356] "Generic (PLEG): container finished" podID="9493df19-2f9e-4514-8cdc-99431b5a0cce" containerID="714c64c2ad156c2b177b06543554f772ab5613b2f622facf204b97b9090b825f" exitCode=0
Jul 29 16:40:36.582289 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.582147    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"
Jul 29 16:40:36.582289 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.582184    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx" event={"ID":"9493df19-2f9e-4514-8cdc-99431b5a0cce","Type":"ContainerDied","Data":"714c64c2ad156c2b177b06543554f772ab5613b2f622facf204b97b9090b825f"}
Jul 29 16:40:36.582289 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.582229    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx" event={"ID":"9493df19-2f9e-4514-8cdc-99431b5a0cce","Type":"ContainerDied","Data":"be1322ea80b77d223f10f89545dd34c73dcac174f10208cc8b643c0457fb37bb"}
Jul 29 16:40:36.582289 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.582256    2560 scope.go:117] "RemoveContainer" containerID="714c64c2ad156c2b177b06543554f772ab5613b2f622facf204b97b9090b825f"
Jul 29 16:40:36.596328 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.596287    2560 scope.go:117] "RemoveContainer" containerID="714c64c2ad156c2b177b06543554f772ab5613b2f622facf204b97b9090b825f"
Jul 29 16:40:36.596518 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.596496    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7p5l\" (UniqueName: \"kubernetes.io/projected/9493df19-2f9e-4514-8cdc-99431b5a0cce-kube-api-access-d7p5l\") pod \"9493df19-2f9e-4514-8cdc-99431b5a0cce\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") "
Jul 29 16:40:36.596597 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.596543    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-dshm\") pod \"9493df19-2f9e-4514-8cdc-99431b5a0cce\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") "
Jul 29 16:40:36.596597 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:40:36.596567    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"714c64c2ad156c2b177b06543554f772ab5613b2f622facf204b97b9090b825f\": container with ID starting with 714c64c2ad156c2b177b06543554f772ab5613b2f622facf204b97b9090b825f not found: ID does not exist" containerID="714c64c2ad156c2b177b06543554f772ab5613b2f622facf204b97b9090b825f"
Jul 29 16:40:36.596678 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.596598    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"714c64c2ad156c2b177b06543554f772ab5613b2f622facf204b97b9090b825f"} err="failed to get container status \"714c64c2ad156c2b177b06543554f772ab5613b2f622facf204b97b9090b825f\": rpc error: code = NotFound desc = could not find container \"714c64c2ad156c2b177b06543554f772ab5613b2f622facf204b97b9090b825f\": container with ID starting with 714c64c2ad156c2b177b06543554f772ab5613b2f622facf204b97b9090b825f not found: ID does not exist"
Jul 29 16:40:36.596678 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.596574    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-model-cache\") pod \"9493df19-2f9e-4514-8cdc-99431b5a0cce\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") "
Jul 29 16:40:36.596778 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.596683    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9493df19-2f9e-4514-8cdc-99431b5a0cce-tls-certs\") pod \"9493df19-2f9e-4514-8cdc-99431b5a0cce\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") "
Jul 29 16:40:36.596778 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.596729    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-home\") pod \"9493df19-2f9e-4514-8cdc-99431b5a0cce\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") "
Jul 29 16:40:36.596859 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.596779    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-model-cache" (OuterVolumeSpecName: "model-cache") pod "9493df19-2f9e-4514-8cdc-99431b5a0cce" (UID: "9493df19-2f9e-4514-8cdc-99431b5a0cce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:40:36.596859 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.596800    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-tmp-dir\") pod \"9493df19-2f9e-4514-8cdc-99431b5a0cce\" (UID: \"9493df19-2f9e-4514-8cdc-99431b5a0cce\") "
Jul 29 16:40:36.597148 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.597118    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-model-cache\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:40:36.597448 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.597383    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9493df19-2f9e-4514-8cdc-99431b5a0cce" (UID: "9493df19-2f9e-4514-8cdc-99431b5a0cce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:40:36.597571 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.597466    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-home" (OuterVolumeSpecName: "home") pod "9493df19-2f9e-4514-8cdc-99431b5a0cce" (UID: "9493df19-2f9e-4514-8cdc-99431b5a0cce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:40:36.598930 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.598882    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-dshm" (OuterVolumeSpecName: "dshm") pod "9493df19-2f9e-4514-8cdc-99431b5a0cce" (UID: "9493df19-2f9e-4514-8cdc-99431b5a0cce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:40:36.599120 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.599098    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9493df19-2f9e-4514-8cdc-99431b5a0cce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9493df19-2f9e-4514-8cdc-99431b5a0cce" (UID: "9493df19-2f9e-4514-8cdc-99431b5a0cce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:40:36.599319 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.599294    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9493df19-2f9e-4514-8cdc-99431b5a0cce-kube-api-access-d7p5l" (OuterVolumeSpecName: "kube-api-access-d7p5l") pod "9493df19-2f9e-4514-8cdc-99431b5a0cce" (UID: "9493df19-2f9e-4514-8cdc-99431b5a0cce"). InnerVolumeSpecName "kube-api-access-d7p5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:40:36.697968 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.697932    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7p5l\" (UniqueName: \"kubernetes.io/projected/9493df19-2f9e-4514-8cdc-99431b5a0cce-kube-api-access-d7p5l\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:40:36.697968 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.697963    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-dshm\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:40:36.697968 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.697973    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9493df19-2f9e-4514-8cdc-99431b5a0cce-tls-certs\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:40:36.698171 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.697982    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-home\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:40:36.698171 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.697991    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9493df19-2f9e-4514-8cdc-99431b5a0cce-tmp-dir\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:40:36.912944 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.912889    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"]
Jul 29 16:40:36.916161 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:36.916135    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-777250/ipp-route-777250-kserve-64489f9659-5qqmx"]
Jul 29 16:40:38.044032 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:40:38.043997    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9493df19-2f9e-4514-8cdc-99431b5a0cce" path="/var/lib/kubelet/pods/9493df19-2f9e-4514-8cdc-99431b5a0cce/volumes"
Jul 29 16:41:15.431956 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.426324    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"]
Jul 29 16:41:15.431956 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.426650    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m" podUID="e3e12246-d240-45ef-8861-19b68ac5f7e4" containerName="payload-processing" containerID="cri-o://f04670750b4216dea2f745f81a5cf9b2e271c046bf33595012232f2c6881f762" gracePeriod=30
Jul 29 16:41:15.438081 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.437315    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"]
Jul 29 16:41:15.438081 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.437597    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8" podUID="625103f6-4a40-4977-9276-a920ed0640b4" containerName="payload-pre-processing" containerID="cri-o://8bd659d2b66a3979e20c10d635c4198ce803d9f05246f414eef8fac741246394" gracePeriod=30
Jul 29 16:41:15.705168 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.705146    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"
Jul 29 16:41:15.708124 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.708104    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"
Jul 29 16:41:15.776638 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.776605    2560 generic.go:356] "Generic (PLEG): container finished" podID="e3e12246-d240-45ef-8861-19b68ac5f7e4" containerID="f04670750b4216dea2f745f81a5cf9b2e271c046bf33595012232f2c6881f762" exitCode=0
Jul 29 16:41:15.776802 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.776650    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"
Jul 29 16:41:15.776802 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.776690    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m" event={"ID":"e3e12246-d240-45ef-8861-19b68ac5f7e4","Type":"ContainerDied","Data":"f04670750b4216dea2f745f81a5cf9b2e271c046bf33595012232f2c6881f762"}
Jul 29 16:41:15.776802 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.776744    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m" event={"ID":"e3e12246-d240-45ef-8861-19b68ac5f7e4","Type":"ContainerDied","Data":"f39069572fd738f309099d373c4dc49ef3a6dc1ce23daf9a232247e48b34875c"}
Jul 29 16:41:15.776802 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.776774    2560 scope.go:117] "RemoveContainer" containerID="f04670750b4216dea2f745f81a5cf9b2e271c046bf33595012232f2c6881f762"
Jul 29 16:41:15.778040 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.778021    2560 generic.go:356] "Generic (PLEG): container finished" podID="625103f6-4a40-4977-9276-a920ed0640b4" containerID="8bd659d2b66a3979e20c10d635c4198ce803d9f05246f414eef8fac741246394" exitCode=0
Jul 29 16:41:15.778129 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.778069    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"
Jul 29 16:41:15.778129 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.778087    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8" event={"ID":"625103f6-4a40-4977-9276-a920ed0640b4","Type":"ContainerDied","Data":"8bd659d2b66a3979e20c10d635c4198ce803d9f05246f414eef8fac741246394"}
Jul 29 16:41:15.778129 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.778121    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8" event={"ID":"625103f6-4a40-4977-9276-a920ed0640b4","Type":"ContainerDied","Data":"102da7f32d8f4a1e5b81c896eb19a8c38bcccf7f83a22d65432e54184056aaaf"}
Jul 29 16:41:15.785975 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.785939    2560 scope.go:117] "RemoveContainer" containerID="f04670750b4216dea2f745f81a5cf9b2e271c046bf33595012232f2c6881f762"
Jul 29 16:41:15.786197 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:41:15.786180    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f04670750b4216dea2f745f81a5cf9b2e271c046bf33595012232f2c6881f762\": container with ID starting with f04670750b4216dea2f745f81a5cf9b2e271c046bf33595012232f2c6881f762 not found: ID does not exist" containerID="f04670750b4216dea2f745f81a5cf9b2e271c046bf33595012232f2c6881f762"
Jul 29 16:41:15.786243 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.786205    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f04670750b4216dea2f745f81a5cf9b2e271c046bf33595012232f2c6881f762"} err="failed to get container status \"f04670750b4216dea2f745f81a5cf9b2e271c046bf33595012232f2c6881f762\": rpc error: code = NotFound desc = could not find container \"f04670750b4216dea2f745f81a5cf9b2e271c046bf33595012232f2c6881f762\": container with ID starting with f04670750b4216dea2f745f81a5cf9b2e271c046bf33595012232f2c6881f762 not found: ID does not exist"
Jul 29 16:41:15.786243 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.786219    2560 scope.go:117] "RemoveContainer" containerID="8bd659d2b66a3979e20c10d635c4198ce803d9f05246f414eef8fac741246394"
Jul 29 16:41:15.796293 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.796272    2560 scope.go:117] "RemoveContainer" containerID="8bd659d2b66a3979e20c10d635c4198ce803d9f05246f414eef8fac741246394"
Jul 29 16:41:15.796557 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:41:15.796537    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bd659d2b66a3979e20c10d635c4198ce803d9f05246f414eef8fac741246394\": container with ID starting with 8bd659d2b66a3979e20c10d635c4198ce803d9f05246f414eef8fac741246394 not found: ID does not exist" containerID="8bd659d2b66a3979e20c10d635c4198ce803d9f05246f414eef8fac741246394"
Jul 29 16:41:15.796624 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.796565    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bd659d2b66a3979e20c10d635c4198ce803d9f05246f414eef8fac741246394"} err="failed to get container status \"8bd659d2b66a3979e20c10d635c4198ce803d9f05246f414eef8fac741246394\": rpc error: code = NotFound desc = could not find container \"8bd659d2b66a3979e20c10d635c4198ce803d9f05246f414eef8fac741246394\": container with ID starting with 8bd659d2b66a3979e20c10d635c4198ce803d9f05246f414eef8fac741246394 not found: ID does not exist"
Jul 29 16:41:15.837139 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.837101    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3e12246-d240-45ef-8861-19b68ac5f7e4-plugins-config-volume\") pod \"e3e12246-d240-45ef-8861-19b68ac5f7e4\" (UID: \"e3e12246-d240-45ef-8861-19b68ac5f7e4\") "
Jul 29 16:41:15.837300 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.837181    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/625103f6-4a40-4977-9276-a920ed0640b4-plugins-config-volume\") pod \"625103f6-4a40-4977-9276-a920ed0640b4\" (UID: \"625103f6-4a40-4977-9276-a920ed0640b4\") "
Jul 29 16:41:15.837300 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.837243    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvphg\" (UniqueName: \"kubernetes.io/projected/625103f6-4a40-4977-9276-a920ed0640b4-kube-api-access-pvphg\") pod \"625103f6-4a40-4977-9276-a920ed0640b4\" (UID: \"625103f6-4a40-4977-9276-a920ed0640b4\") "
Jul 29 16:41:15.837300 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.837258    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4zh9\" (UniqueName: \"kubernetes.io/projected/e3e12246-d240-45ef-8861-19b68ac5f7e4-kube-api-access-p4zh9\") pod \"e3e12246-d240-45ef-8861-19b68ac5f7e4\" (UID: \"e3e12246-d240-45ef-8861-19b68ac5f7e4\") "
Jul 29 16:41:15.837558 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.837529    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3e12246-d240-45ef-8861-19b68ac5f7e4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e3e12246-d240-45ef-8861-19b68ac5f7e4" (UID: "e3e12246-d240-45ef-8861-19b68ac5f7e4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:41:15.837558 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.837552    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/625103f6-4a40-4977-9276-a920ed0640b4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "625103f6-4a40-4977-9276-a920ed0640b4" (UID: "625103f6-4a40-4977-9276-a920ed0640b4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:41:15.839339 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.839312    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3e12246-d240-45ef-8861-19b68ac5f7e4-kube-api-access-p4zh9" (OuterVolumeSpecName: "kube-api-access-p4zh9") pod "e3e12246-d240-45ef-8861-19b68ac5f7e4" (UID: "e3e12246-d240-45ef-8861-19b68ac5f7e4"). InnerVolumeSpecName "kube-api-access-p4zh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:41:15.839453 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.839400    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/625103f6-4a40-4977-9276-a920ed0640b4-kube-api-access-pvphg" (OuterVolumeSpecName: "kube-api-access-pvphg") pod "625103f6-4a40-4977-9276-a920ed0640b4" (UID: "625103f6-4a40-4977-9276-a920ed0640b4"). InnerVolumeSpecName "kube-api-access-pvphg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:41:15.938197 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.938162    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvphg\" (UniqueName: \"kubernetes.io/projected/625103f6-4a40-4977-9276-a920ed0640b4-kube-api-access-pvphg\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:41:15.938197 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.938191    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4zh9\" (UniqueName: \"kubernetes.io/projected/e3e12246-d240-45ef-8861-19b68ac5f7e4-kube-api-access-p4zh9\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:41:15.938197 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.938201    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3e12246-d240-45ef-8861-19b68ac5f7e4-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:41:15.938423 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:15.938211    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/625103f6-4a40-4977-9276-a920ed0640b4-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:41:16.093631 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:16.093601    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"]
Jul 29 16:41:16.096830 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:16.096807    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3c2006-585d5dbf6-4j29m"]
Jul 29 16:41:16.107507 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:16.107479    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"]
Jul 29 16:41:16.112941 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:16.112900    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-3c2006-84665db99b-9fsh8"]
Jul 29 16:41:18.043031 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:18.042990    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="625103f6-4a40-4977-9276-a920ed0640b4" path="/var/lib/kubelet/pods/625103f6-4a40-4977-9276-a920ed0640b4/volumes"
Jul 29 16:41:18.043545 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:18.043526    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3e12246-d240-45ef-8861-19b68ac5f7e4" path="/var/lib/kubelet/pods/e3e12246-d240-45ef-8861-19b68ac5f7e4/volumes"
Jul 29 16:41:18.561329 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:18.561296    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:41:33.145018 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.144982    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"]
Jul 29 16:41:33.151280 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.151246    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"]
Jul 29 16:41:33.151521 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.151474    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq" podUID="86b4617f-47ab-4ccc-aced-b2892a8b956e" containerName="payload-pre-processing" containerID="cri-o://df63bd664e8615ba34ccd1ce27a0bbb24c1f5c956e5bab159a6306ff05e1b207" gracePeriod=30
Jul 29 16:41:33.424683 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.424657    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"
Jul 29 16:41:33.478897 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.478869    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sg6m\" (UniqueName: \"kubernetes.io/projected/86b4617f-47ab-4ccc-aced-b2892a8b956e-kube-api-access-9sg6m\") pod \"86b4617f-47ab-4ccc-aced-b2892a8b956e\" (UID: \"86b4617f-47ab-4ccc-aced-b2892a8b956e\") "
Jul 29 16:41:33.479092 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.478951    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86b4617f-47ab-4ccc-aced-b2892a8b956e-plugins-config-volume\") pod \"86b4617f-47ab-4ccc-aced-b2892a8b956e\" (UID: \"86b4617f-47ab-4ccc-aced-b2892a8b956e\") "
Jul 29 16:41:33.479338 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.479311    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86b4617f-47ab-4ccc-aced-b2892a8b956e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "86b4617f-47ab-4ccc-aced-b2892a8b956e" (UID: "86b4617f-47ab-4ccc-aced-b2892a8b956e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:41:33.480868 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.480846    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86b4617f-47ab-4ccc-aced-b2892a8b956e-kube-api-access-9sg6m" (OuterVolumeSpecName: "kube-api-access-9sg6m") pod "86b4617f-47ab-4ccc-aced-b2892a8b956e" (UID: "86b4617f-47ab-4ccc-aced-b2892a8b956e"). InnerVolumeSpecName "kube-api-access-9sg6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:41:33.579512 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.579472    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sg6m\" (UniqueName: \"kubernetes.io/projected/86b4617f-47ab-4ccc-aced-b2892a8b956e-kube-api-access-9sg6m\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:41:33.579512 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.579507    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86b4617f-47ab-4ccc-aced-b2892a8b956e-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:41:33.867747 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.867653    2560 generic.go:356] "Generic (PLEG): container finished" podID="86b4617f-47ab-4ccc-aced-b2892a8b956e" containerID="df63bd664e8615ba34ccd1ce27a0bbb24c1f5c956e5bab159a6306ff05e1b207" exitCode=0
Jul 29 16:41:33.867747 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.867699    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"
Jul 29 16:41:33.867747 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.867732    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq" event={"ID":"86b4617f-47ab-4ccc-aced-b2892a8b956e","Type":"ContainerDied","Data":"df63bd664e8615ba34ccd1ce27a0bbb24c1f5c956e5bab159a6306ff05e1b207"}
Jul 29 16:41:33.868076 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.867795    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq" event={"ID":"86b4617f-47ab-4ccc-aced-b2892a8b956e","Type":"ContainerDied","Data":"c92167b7ce9ca1c4d03da2434d91edbb69be4645e222045753282b66d40cdf85"}
Jul 29 16:41:33.868076 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.867826    2560 scope.go:117] "RemoveContainer" containerID="df63bd664e8615ba34ccd1ce27a0bbb24c1f5c956e5bab159a6306ff05e1b207"
Jul 29 16:41:33.868076 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.868060    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb" podUID="580c020b-a6cd-4185-92f2-f3184defdca2" containerName="payload-processing" containerID="cri-o://eec05a244b29a1cf6853fa8bec819e81b5a480813b7c4eca66fd36fb292a4720" gracePeriod=30
Jul 29 16:41:33.877264 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.877115    2560 scope.go:117] "RemoveContainer" containerID="df63bd664e8615ba34ccd1ce27a0bbb24c1f5c956e5bab159a6306ff05e1b207"
Jul 29 16:41:33.877409 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:41:33.877372    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df63bd664e8615ba34ccd1ce27a0bbb24c1f5c956e5bab159a6306ff05e1b207\": container with ID starting with df63bd664e8615ba34ccd1ce27a0bbb24c1f5c956e5bab159a6306ff05e1b207 not found: ID does not exist" containerID="df63bd664e8615ba34ccd1ce27a0bbb24c1f5c956e5bab159a6306ff05e1b207"
Jul 29 16:41:33.877458 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.877405    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df63bd664e8615ba34ccd1ce27a0bbb24c1f5c956e5bab159a6306ff05e1b207"} err="failed to get container status \"df63bd664e8615ba34ccd1ce27a0bbb24c1f5c956e5bab159a6306ff05e1b207\": rpc error: code = NotFound desc = could not find container \"df63bd664e8615ba34ccd1ce27a0bbb24c1f5c956e5bab159a6306ff05e1b207\": container with ID starting with df63bd664e8615ba34ccd1ce27a0bbb24c1f5c956e5bab159a6306ff05e1b207 not found: ID does not exist"
Jul 29 16:41:33.939984 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.939953    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"]
Jul 29 16:41:33.944202 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:33.944177    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-81ab8b-66b69c6d7b-zblwq"]
Jul 29 16:41:34.044598 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.044565    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86b4617f-47ab-4ccc-aced-b2892a8b956e" path="/var/lib/kubelet/pods/86b4617f-47ab-4ccc-aced-b2892a8b956e/volumes"
Jul 29 16:41:34.113528 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.113504    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"
Jul 29 16:41:34.184999 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.184971    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qn6m4\" (UniqueName: \"kubernetes.io/projected/580c020b-a6cd-4185-92f2-f3184defdca2-kube-api-access-qn6m4\") pod \"580c020b-a6cd-4185-92f2-f3184defdca2\" (UID: \"580c020b-a6cd-4185-92f2-f3184defdca2\") "
Jul 29 16:41:34.185333 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.185166    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/580c020b-a6cd-4185-92f2-f3184defdca2-plugins-config-volume\") pod \"580c020b-a6cd-4185-92f2-f3184defdca2\" (UID: \"580c020b-a6cd-4185-92f2-f3184defdca2\") "
Jul 29 16:41:34.185536 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.185506    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/580c020b-a6cd-4185-92f2-f3184defdca2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "580c020b-a6cd-4185-92f2-f3184defdca2" (UID: "580c020b-a6cd-4185-92f2-f3184defdca2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:41:34.187099 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.187077    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/580c020b-a6cd-4185-92f2-f3184defdca2-kube-api-access-qn6m4" (OuterVolumeSpecName: "kube-api-access-qn6m4") pod "580c020b-a6cd-4185-92f2-f3184defdca2" (UID: "580c020b-a6cd-4185-92f2-f3184defdca2"). InnerVolumeSpecName "kube-api-access-qn6m4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:41:34.286188 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.286157    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/580c020b-a6cd-4185-92f2-f3184defdca2-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:41:34.286188 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.286184    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qn6m4\" (UniqueName: \"kubernetes.io/projected/580c020b-a6cd-4185-92f2-f3184defdca2-kube-api-access-qn6m4\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:41:34.875037 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.875003    2560 generic.go:356] "Generic (PLEG): container finished" podID="580c020b-a6cd-4185-92f2-f3184defdca2" containerID="eec05a244b29a1cf6853fa8bec819e81b5a480813b7c4eca66fd36fb292a4720" exitCode=0
Jul 29 16:41:34.875219 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.875056    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb" event={"ID":"580c020b-a6cd-4185-92f2-f3184defdca2","Type":"ContainerDied","Data":"eec05a244b29a1cf6853fa8bec819e81b5a480813b7c4eca66fd36fb292a4720"}
Jul 29 16:41:34.875219 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.875066    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"
Jul 29 16:41:34.875219 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.875096    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb" event={"ID":"580c020b-a6cd-4185-92f2-f3184defdca2","Type":"ContainerDied","Data":"4fa0fb23932bca72cbec11a1040c41dbee24b40c600be7a296ce4deb4c07c618"}
Jul 29 16:41:34.875219 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.875118    2560 scope.go:117] "RemoveContainer" containerID="eec05a244b29a1cf6853fa8bec819e81b5a480813b7c4eca66fd36fb292a4720"
Jul 29 16:41:34.884020 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.883997    2560 scope.go:117] "RemoveContainer" containerID="eec05a244b29a1cf6853fa8bec819e81b5a480813b7c4eca66fd36fb292a4720"
Jul 29 16:41:34.884268 ip-10-0-143-118 kubenswrapper[2560]: E0729 16:41:34.884247    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eec05a244b29a1cf6853fa8bec819e81b5a480813b7c4eca66fd36fb292a4720\": container with ID starting with eec05a244b29a1cf6853fa8bec819e81b5a480813b7c4eca66fd36fb292a4720 not found: ID does not exist" containerID="eec05a244b29a1cf6853fa8bec819e81b5a480813b7c4eca66fd36fb292a4720"
Jul 29 16:41:34.884327 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.884275    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eec05a244b29a1cf6853fa8bec819e81b5a480813b7c4eca66fd36fb292a4720"} err="failed to get container status \"eec05a244b29a1cf6853fa8bec819e81b5a480813b7c4eca66fd36fb292a4720\": rpc error: code = NotFound desc = could not find container \"eec05a244b29a1cf6853fa8bec819e81b5a480813b7c4eca66fd36fb292a4720\": container with ID starting with eec05a244b29a1cf6853fa8bec819e81b5a480813b7c4eca66fd36fb292a4720 not found: ID does not exist"
Jul 29 16:41:34.897032 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.896998    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"]
Jul 29 16:41:34.900738 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:34.900715    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-81ab8b-c96fcf9fd-jmvkb"]
Jul 29 16:41:36.045861 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:36.045821    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="580c020b-a6cd-4185-92f2-f3184defdca2" path="/var/lib/kubelet/pods/580c020b-a6cd-4185-92f2-f3184defdca2/volumes"
Jul 29 16:41:36.349359 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:36.349271    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:41:50.754241 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:50.754200    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"]
Jul 29 16:41:50.754742 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:50.754386    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v" podUID="ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab" containerName="payload-processing" containerID="cri-o://a5cb10ed1f1d84f1b0d9f8f1042ab9ed6a6abbcdbdcdfd969aa3dd7426a0858b" gracePeriod=30
Jul 29 16:41:50.763106 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:50.761658    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"]
Jul 29 16:41:50.763106 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:50.761901    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr" podUID="b227d330-6988-4cb3-a5da-233000216a14" containerName="payload-pre-processing" containerID="cri-o://83e6523a0ee5183bf44f5b6efdadd593272d6a8dda9f99167af3f2821345356d" gracePeriod=30
Jul 29 16:41:50.953078 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:50.953046    2560 generic.go:356] "Generic (PLEG): container finished" podID="ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab" containerID="a5cb10ed1f1d84f1b0d9f8f1042ab9ed6a6abbcdbdcdfd969aa3dd7426a0858b" exitCode=0
Jul 29 16:41:50.953347 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:50.953256    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v" event={"ID":"ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab","Type":"ContainerDied","Data":"a5cb10ed1f1d84f1b0d9f8f1042ab9ed6a6abbcdbdcdfd969aa3dd7426a0858b"}
Jul 29 16:41:50.954967 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:50.954909    2560 generic.go:356] "Generic (PLEG): container finished" podID="b227d330-6988-4cb3-a5da-233000216a14" containerID="83e6523a0ee5183bf44f5b6efdadd593272d6a8dda9f99167af3f2821345356d" exitCode=0
Jul 29 16:41:50.955102 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:50.955045    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr" event={"ID":"b227d330-6988-4cb3-a5da-233000216a14","Type":"ContainerDied","Data":"83e6523a0ee5183bf44f5b6efdadd593272d6a8dda9f99167af3f2821345356d"}
Jul 29 16:41:51.033688 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.033668    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"
Jul 29 16:41:51.036476 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.036459    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"
Jul 29 16:41:51.137833 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.137803    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24m79\" (UniqueName: \"kubernetes.io/projected/ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab-kube-api-access-24m79\") pod \"ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab\" (UID: \"ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab\") "
Jul 29 16:41:51.138035 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.137848    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npd9x\" (UniqueName: \"kubernetes.io/projected/b227d330-6988-4cb3-a5da-233000216a14-kube-api-access-npd9x\") pod \"b227d330-6988-4cb3-a5da-233000216a14\" (UID: \"b227d330-6988-4cb3-a5da-233000216a14\") "
Jul 29 16:41:51.138035 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.137883    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b227d330-6988-4cb3-a5da-233000216a14-plugins-config-volume\") pod \"b227d330-6988-4cb3-a5da-233000216a14\" (UID: \"b227d330-6988-4cb3-a5da-233000216a14\") "
Jul 29 16:41:51.138035 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.137897    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab-plugins-config-volume\") pod \"ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab\" (UID: \"ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab\") "
Jul 29 16:41:51.138341 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.138305    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b227d330-6988-4cb3-a5da-233000216a14-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b227d330-6988-4cb3-a5da-233000216a14" (UID: "b227d330-6988-4cb3-a5da-233000216a14"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:41:51.138477 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.138389    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab" (UID: "ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:41:51.140041 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.140015    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b227d330-6988-4cb3-a5da-233000216a14-kube-api-access-npd9x" (OuterVolumeSpecName: "kube-api-access-npd9x") pod "b227d330-6988-4cb3-a5da-233000216a14" (UID: "b227d330-6988-4cb3-a5da-233000216a14"). InnerVolumeSpecName "kube-api-access-npd9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:41:51.140124 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.140054    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab-kube-api-access-24m79" (OuterVolumeSpecName: "kube-api-access-24m79") pod "ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab" (UID: "ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab"). InnerVolumeSpecName "kube-api-access-24m79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:41:51.238970 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.238945    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24m79\" (UniqueName: \"kubernetes.io/projected/ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab-kube-api-access-24m79\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:41:51.238970 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.238969    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npd9x\" (UniqueName: \"kubernetes.io/projected/b227d330-6988-4cb3-a5da-233000216a14-kube-api-access-npd9x\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:41:51.239131 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.238977    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b227d330-6988-4cb3-a5da-233000216a14-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:41:51.239131 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.238986    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab-plugins-config-volume\") on node \"ip-10-0-143-118.ec2.internal\" DevicePath \"\""
Jul 29 16:41:51.961072 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.961044    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"
Jul 29 16:41:51.961072 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.961061    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v" event={"ID":"ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab","Type":"ContainerDied","Data":"6396adcda47df8bea68092255fbe8ff14f08bad722106bdd029edef2a52d86df"}
Jul 29 16:41:51.961583 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.961122    2560 scope.go:117] "RemoveContainer" containerID="a5cb10ed1f1d84f1b0d9f8f1042ab9ed6a6abbcdbdcdfd969aa3dd7426a0858b"
Jul 29 16:41:51.962567 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.962545    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr" event={"ID":"b227d330-6988-4cb3-a5da-233000216a14","Type":"ContainerDied","Data":"f4b672cc4b517ee35c3ba1c8ebd29aa372782687e27b706ff105a6753fc0497f"}
Jul 29 16:41:51.962669 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.962600    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"
Jul 29 16:41:51.969717 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.969701    2560 scope.go:117] "RemoveContainer" containerID="83e6523a0ee5183bf44f5b6efdadd593272d6a8dda9f99167af3f2821345356d"
Jul 29 16:41:51.986168 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.986147    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"]
Jul 29 16:41:51.990341 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:51.990322    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-eccd34-94f974d4d-lgjkr"]
Jul 29 16:41:52.002501 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:52.002482    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"]
Jul 29 16:41:52.006248 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:52.006226    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-eccd34-f454cc49d-skn5v"]
Jul 29 16:41:52.042192 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:52.042123    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b227d330-6988-4cb3-a5da-233000216a14" path="/var/lib/kubelet/pods/b227d330-6988-4cb3-a5da-233000216a14/volumes"
Jul 29 16:41:52.042565 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:52.042546    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab" path="/var/lib/kubelet/pods/ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab/volumes"
Jul 29 16:41:53.355791 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:41:53.355761    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fmhmp"]
Jul 29 16:42:04.468948 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:04.468854    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75b794959f-zwzb7_7238c72e-3821-425c-8c55-e7e65ac8d3da/copy-manifests/0.log"
Jul 29 16:42:04.488862 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:04.488836    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75b794959f-zwzb7_7238c72e-3821-425c-8c55-e7e65ac8d3da/manager/0.log"
Jul 29 16:42:06.173343 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:06.173317    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt_147ac01f-e082-4b0a-8113-789f6585b6c6/util/0.log"
Jul 29 16:42:06.180210 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:06.180189    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt_147ac01f-e082-4b0a-8113-789f6585b6c6/pull/0.log"
Jul 29 16:42:06.186852 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:06.186836    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt_147ac01f-e082-4b0a-8113-789f6585b6c6/extract/0.log"
Jul 29 16:42:06.883793 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:06.883766    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-cq2xp_f5e90aa8-ef73-4060-94e1-6a1d987856c1/kuadrant-console-plugin/0.log"
Jul 29 16:42:07.244745 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:07.244713    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-fmhmp_e959de9a-56d3-4856-9c2b-37fac34a8878/limitador/0.log"
Jul 29 16:42:07.944466 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:07.944439    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86bbb74d46-5qf7q_2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf/kube-auth-proxy/0.log"
Jul 29 16:42:09.193870 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:09.193842    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f76fhk_e8d1a803-de32-4561-b655-0cca8548e992/main/0.log"
Jul 29 16:42:09.419665 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:09.419638    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-wfbpr_467eecae-082a-4f1f-a632-bd943cb45e5d/main/0.log"
Jul 29 16:42:14.874406 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.874368    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kqvb7/must-gather-smdw2"]
Jul 29 16:42:14.874874 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.874851    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b227d330-6988-4cb3-a5da-233000216a14" containerName="payload-pre-processing"
Jul 29 16:42:14.874976 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.874878    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b227d330-6988-4cb3-a5da-233000216a14" containerName="payload-pre-processing"
Jul 29 16:42:14.874976 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.874890    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63bb369b-b9dc-4f63-8c1e-e1728fdefe83" containerName="payload-processing"
Jul 29 16:42:14.874976 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.874899    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="63bb369b-b9dc-4f63-8c1e-e1728fdefe83" containerName="payload-processing"
Jul 29 16:42:14.874976 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.874911    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c56b9835-dc19-4e16-a04d-83faa1afdec9" containerName="payload-pre-processing"
Jul 29 16:42:14.874976 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.874938    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c56b9835-dc19-4e16-a04d-83faa1afdec9" containerName="payload-pre-processing"
Jul 29 16:42:14.874976 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.874955    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="625103f6-4a40-4977-9276-a920ed0640b4" containerName="payload-pre-processing"
Jul 29 16:42:14.874976 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.874963    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="625103f6-4a40-4977-9276-a920ed0640b4" containerName="payload-pre-processing"
Jul 29 16:42:14.874976 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.874974    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="580c020b-a6cd-4185-92f2-f3184defdca2" containerName="payload-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.874982    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="580c020b-a6cd-4185-92f2-f3184defdca2" containerName="payload-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.874993    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc3573cc-cbb7-44eb-8192-aa8550053190" containerName="payload-pre-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875002    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc3573cc-cbb7-44eb-8192-aa8550053190" containerName="payload-pre-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875012    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3e12246-d240-45ef-8861-19b68ac5f7e4" containerName="payload-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875021    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3e12246-d240-45ef-8861-19b68ac5f7e4" containerName="payload-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875035    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86b4617f-47ab-4ccc-aced-b2892a8b956e" containerName="payload-pre-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875043    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="86b4617f-47ab-4ccc-aced-b2892a8b956e" containerName="payload-pre-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875064    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9493df19-2f9e-4514-8cdc-99431b5a0cce" containerName="main"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875072    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9493df19-2f9e-4514-8cdc-99431b5a0cce" containerName="main"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875084    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab" containerName="payload-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875092    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab" containerName="payload-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875189    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="63bb369b-b9dc-4f63-8c1e-e1728fdefe83" containerName="payload-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875203    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="86b4617f-47ab-4ccc-aced-b2892a8b956e" containerName="payload-pre-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875212    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecbfc11e-3ca6-4e76-bb86-cb35a9db83ab" containerName="payload-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875221    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b227d330-6988-4cb3-a5da-233000216a14" containerName="payload-pre-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875231    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="580c020b-a6cd-4185-92f2-f3184defdca2" containerName="payload-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875242    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc3573cc-cbb7-44eb-8192-aa8550053190" containerName="payload-pre-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875251    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="625103f6-4a40-4977-9276-a920ed0640b4" containerName="payload-pre-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875261    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c56b9835-dc19-4e16-a04d-83faa1afdec9" containerName="payload-pre-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875270    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3e12246-d240-45ef-8861-19b68ac5f7e4" containerName="payload-processing"
Jul 29 16:42:14.875388 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.875278    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9493df19-2f9e-4514-8cdc-99431b5a0cce" containerName="main"
Jul 29 16:42:14.880160 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.880139    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kqvb7/must-gather-smdw2"
Jul 29 16:42:14.883430 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.883408    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kqvb7\"/\"kube-root-ca.crt\""
Jul 29 16:42:14.883555 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.883414    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kqvb7\"/\"default-dockercfg-hb6xm\""
Jul 29 16:42:14.884700 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.884683    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kqvb7\"/\"openshift-service-ca.crt\""
Jul 29 16:42:14.889741 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.889722    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kqvb7/must-gather-smdw2"]
Jul 29 16:42:14.943443 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.943409    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/da5c9810-ec49-4f0a-9a01-01a11fef1c18-must-gather-output\") pod \"must-gather-smdw2\" (UID: \"da5c9810-ec49-4f0a-9a01-01a11fef1c18\") " pod="openshift-must-gather-kqvb7/must-gather-smdw2"
Jul 29 16:42:14.943600 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:14.943450    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z94ml\" (UniqueName: \"kubernetes.io/projected/da5c9810-ec49-4f0a-9a01-01a11fef1c18-kube-api-access-z94ml\") pod \"must-gather-smdw2\" (UID: \"da5c9810-ec49-4f0a-9a01-01a11fef1c18\") " pod="openshift-must-gather-kqvb7/must-gather-smdw2"
Jul 29 16:42:15.043758 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:15.043723    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/da5c9810-ec49-4f0a-9a01-01a11fef1c18-must-gather-output\") pod \"must-gather-smdw2\" (UID: \"da5c9810-ec49-4f0a-9a01-01a11fef1c18\") " pod="openshift-must-gather-kqvb7/must-gather-smdw2"
Jul 29 16:42:15.043758 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:15.043758    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z94ml\" (UniqueName: \"kubernetes.io/projected/da5c9810-ec49-4f0a-9a01-01a11fef1c18-kube-api-access-z94ml\") pod \"must-gather-smdw2\" (UID: \"da5c9810-ec49-4f0a-9a01-01a11fef1c18\") " pod="openshift-must-gather-kqvb7/must-gather-smdw2"
Jul 29 16:42:15.044076 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:15.044057    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/da5c9810-ec49-4f0a-9a01-01a11fef1c18-must-gather-output\") pod \"must-gather-smdw2\" (UID: \"da5c9810-ec49-4f0a-9a01-01a11fef1c18\") " pod="openshift-must-gather-kqvb7/must-gather-smdw2"
Jul 29 16:42:15.052000 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:15.051973    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z94ml\" (UniqueName: \"kubernetes.io/projected/da5c9810-ec49-4f0a-9a01-01a11fef1c18-kube-api-access-z94ml\") pod \"must-gather-smdw2\" (UID: \"da5c9810-ec49-4f0a-9a01-01a11fef1c18\") " pod="openshift-must-gather-kqvb7/must-gather-smdw2"
Jul 29 16:42:15.190348 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:15.190311    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kqvb7/must-gather-smdw2"
Jul 29 16:42:15.321331 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:15.321302    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kqvb7/must-gather-smdw2"]
Jul 29 16:42:15.323485 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:42:15.323450    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda5c9810_ec49_4f0a_9a01_01a11fef1c18.slice/crio-541b3fd363bb578d85ccc84fb79943ae64753fb7fdb860268007d0e67c8ad0cc WatchSource:0}: Error finding container 541b3fd363bb578d85ccc84fb79943ae64753fb7fdb860268007d0e67c8ad0cc: Status 404 returned error can't find the container with id 541b3fd363bb578d85ccc84fb79943ae64753fb7fdb860268007d0e67c8ad0cc
Jul 29 16:42:15.325258 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:15.325240    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 16:42:16.081276 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:16.081239    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kqvb7/must-gather-smdw2" event={"ID":"da5c9810-ec49-4f0a-9a01-01a11fef1c18","Type":"ContainerStarted","Data":"541b3fd363bb578d85ccc84fb79943ae64753fb7fdb860268007d0e67c8ad0cc"}
Jul 29 16:42:17.089203 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:17.089168    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kqvb7/must-gather-smdw2" event={"ID":"da5c9810-ec49-4f0a-9a01-01a11fef1c18","Type":"ContainerStarted","Data":"ff34b37f51511d9e59c2d44bdcd6323aa52625d474a11cdb0c98f81d153dfe7e"}
Jul 29 16:42:17.089637 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:17.089214    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kqvb7/must-gather-smdw2" event={"ID":"da5c9810-ec49-4f0a-9a01-01a11fef1c18","Type":"ContainerStarted","Data":"dfb39ca5f4085e4c5b946536f4b8b241aa6ec23b230a7226b2f5fe4e741cd2cc"}
Jul 29 16:42:17.110539 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:17.110475    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kqvb7/must-gather-smdw2" podStartSLOduration=2.185609122 podStartE2EDuration="3.110453025s" totalImagesPullingTime="924.843903ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:42:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:42:17.10868894 +0000 UTC m=+3341.550895200" watchObservedRunningTime="2026-07-29 16:42:17.110453025 +0000 UTC m=+3341.552659285"
Jul 29 16:42:17.812813 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:17.812779    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6w8mg_fce7281a-8d9e-4cda-a8ba-7480f20da3ef/global-pull-secret-syncer/0.log"
Jul 29 16:42:17.983638 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:17.983590    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rghwf_78d0a524-d0ac-4fc8-a3fa-daa48cd2edb1/konnectivity-agent/0.log"
Jul 29 16:42:18.081848 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:18.081778    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-118.ec2.internal_cf62d983a3713d310961fc20664891a3/haproxy/0.log"
Jul 29 16:42:22.632747 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:22.632685    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt_147ac01f-e082-4b0a-8113-789f6585b6c6/extract/0.log"
Jul 29 16:42:22.666199 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:22.666159    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt_147ac01f-e082-4b0a-8113-789f6585b6c6/util/0.log"
Jul 29 16:42:22.695562 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:22.695529    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kfplt_147ac01f-e082-4b0a-8113-789f6585b6c6/pull/0.log"
Jul 29 16:42:23.604647 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:23.604591    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-cq2xp_f5e90aa8-ef73-4060-94e1-6a1d987856c1/kuadrant-console-plugin/0.log"
Jul 29 16:42:24.010461 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:24.010434    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-fmhmp_e959de9a-56d3-4856-9c2b-37fac34a8878/limitador/0.log"
Jul 29 16:42:25.784022 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:25.783964    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_521c5283-7731-46ee-aa52-0e7057d2d5bb/alertmanager/0.log"
Jul 29 16:42:25.812184 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:25.812152    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_521c5283-7731-46ee-aa52-0e7057d2d5bb/config-reloader/0.log"
Jul 29 16:42:25.833464 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:25.833419    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_521c5283-7731-46ee-aa52-0e7057d2d5bb/kube-rbac-proxy-web/0.log"
Jul 29 16:42:25.854261 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:25.854215    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_521c5283-7731-46ee-aa52-0e7057d2d5bb/kube-rbac-proxy/0.log"
Jul 29 16:42:25.879220 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:25.879190    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_521c5283-7731-46ee-aa52-0e7057d2d5bb/kube-rbac-proxy-metric/0.log"
Jul 29 16:42:25.899325 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:25.899289    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_521c5283-7731-46ee-aa52-0e7057d2d5bb/prom-label-proxy/0.log"
Jul 29 16:42:25.917652 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:25.917625    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_521c5283-7731-46ee-aa52-0e7057d2d5bb/init-config-reloader/0.log"
Jul 29 16:42:26.059849 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:26.059777    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-b85954cdf-gnmfg_e0356d1c-c664-4886-afab-0dda476cef3b/metrics-server/0.log"
Jul 29 16:42:26.188638 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:26.188609    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s5l4t_35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11/node-exporter/0.log"
Jul 29 16:42:26.207664 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:26.207636    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s5l4t_35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11/kube-rbac-proxy/0.log"
Jul 29 16:42:26.225194 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:26.225164    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s5l4t_35eb4f03-4ed5-4bf6-9b47-ce47e30f9d11/init-textfile/0.log"
Jul 29 16:42:26.316261 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:26.316188    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-5mlq9_91fc3e7a-d563-45f7-a927-874c9f3ac6df/kube-rbac-proxy-main/0.log"
Jul 29 16:42:26.334927 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:26.334891    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-5mlq9_91fc3e7a-d563-45f7-a927-874c9f3ac6df/kube-rbac-proxy-self/0.log"
Jul 29 16:42:26.352854 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:26.352824    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-5mlq9_91fc3e7a-d563-45f7-a927-874c9f3ac6df/openshift-state-metrics/0.log"
Jul 29 16:42:26.575750 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:26.575674    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-4xfvm_13ca2145-9969-4721-a676-86ce906270c3/prometheus-operator-admission-webhook/0.log"
Jul 29 16:42:26.686148 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:26.686118    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d95c9f985-z99zd_d88bed02-586a-40a1-babf-b48a43f918a7/thanos-query/0.log"
Jul 29 16:42:26.704135 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:26.704105    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d95c9f985-z99zd_d88bed02-586a-40a1-babf-b48a43f918a7/kube-rbac-proxy-web/0.log"
Jul 29 16:42:26.723488 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:26.723460    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d95c9f985-z99zd_d88bed02-586a-40a1-babf-b48a43f918a7/kube-rbac-proxy/0.log"
Jul 29 16:42:26.741129 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:26.741101    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d95c9f985-z99zd_d88bed02-586a-40a1-babf-b48a43f918a7/prom-label-proxy/0.log"
Jul 29 16:42:26.759295 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:26.759268    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d95c9f985-z99zd_d88bed02-586a-40a1-babf-b48a43f918a7/kube-rbac-proxy-rules/0.log"
Jul 29 16:42:26.776963 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:26.776938    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d95c9f985-z99zd_d88bed02-586a-40a1-babf-b48a43f918a7/kube-rbac-proxy-metrics/0.log"
Jul 29 16:42:27.894199 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:27.894152    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"]
Jul 29 16:42:27.898699 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:27.898673    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:27.912488 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:27.912455    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"]
Jul 29 16:42:27.970392 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:27.970352    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e0dd4eea-0d71-400d-99b7-0970d6406570-podres\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:27.970699 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:27.970677    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8jv5\" (UniqueName: \"kubernetes.io/projected/e0dd4eea-0d71-400d-99b7-0970d6406570-kube-api-access-g8jv5\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:27.970895 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:27.970880    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0dd4eea-0d71-400d-99b7-0970d6406570-sys\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:27.971065 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:27.971047    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0dd4eea-0d71-400d-99b7-0970d6406570-lib-modules\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:27.971204 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:27.971190    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e0dd4eea-0d71-400d-99b7-0970d6406570-proc\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:28.072705 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:28.072664    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e0dd4eea-0d71-400d-99b7-0970d6406570-proc\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:28.072895 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:28.072831    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e0dd4eea-0d71-400d-99b7-0970d6406570-podres\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:28.072895 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:28.072868    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8jv5\" (UniqueName: \"kubernetes.io/projected/e0dd4eea-0d71-400d-99b7-0970d6406570-kube-api-access-g8jv5\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:28.073053 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:28.073000    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0dd4eea-0d71-400d-99b7-0970d6406570-sys\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:28.073053 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:28.073031    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0dd4eea-0d71-400d-99b7-0970d6406570-lib-modules\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:28.073234 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:28.073211    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0dd4eea-0d71-400d-99b7-0970d6406570-lib-modules\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:28.073302 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:28.073284    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e0dd4eea-0d71-400d-99b7-0970d6406570-proc\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:28.073600 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:28.073581    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e0dd4eea-0d71-400d-99b7-0970d6406570-podres\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:28.074574 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:28.074544    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0dd4eea-0d71-400d-99b7-0970d6406570-sys\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:28.085499 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:28.085473    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8jv5\" (UniqueName: \"kubernetes.io/projected/e0dd4eea-0d71-400d-99b7-0970d6406570-kube-api-access-g8jv5\") pod \"perf-node-gather-daemonset-jswkp\" (UID: \"e0dd4eea-0d71-400d-99b7-0970d6406570\") " pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:28.213404 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:28.213304    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:28.396314 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:28.396214    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"]
Jul 29 16:42:28.405732 ip-10-0-143-118 kubenswrapper[2560]: W0729 16:42:28.405700    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode0dd4eea_0d71_400d_99b7_0970d6406570.slice/crio-b340d76248d7491cd8cd1798989b93002b2f256c85bd38c78f3cde6a09f3fd92 WatchSource:0}: Error finding container b340d76248d7491cd8cd1798989b93002b2f256c85bd38c78f3cde6a09f3fd92: Status 404 returned error can't find the container with id b340d76248d7491cd8cd1798989b93002b2f256c85bd38c78f3cde6a09f3fd92
Jul 29 16:42:29.085628 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:29.085596    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-gdtj7_30ed7de3-8456-43fe-94ea-82e6dbfc4a0b/download-server/0.log"
Jul 29 16:42:29.182465 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:29.182428    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp" event={"ID":"e0dd4eea-0d71-400d-99b7-0970d6406570","Type":"ContainerStarted","Data":"3d257335f0d8274e8c626439a0cd65638445f2930862fdd6407c53def64cf17c"}
Jul 29 16:42:29.182644 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:29.182489    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp" event={"ID":"e0dd4eea-0d71-400d-99b7-0970d6406570","Type":"ContainerStarted","Data":"b340d76248d7491cd8cd1798989b93002b2f256c85bd38c78f3cde6a09f3fd92"}
Jul 29 16:42:29.182756 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:29.182732    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:29.198707 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:29.198656    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp" podStartSLOduration=2.198636377 podStartE2EDuration="2.198636377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 16:42:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:42:29.198607048 +0000 UTC m=+3353.640813310" watchObservedRunningTime="2026-07-29 16:42:29.198636377 +0000 UTC m=+3353.640842636"
Jul 29 16:42:30.612995 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:30.612963    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mszm6_8aac57b2-7e6a-45ec-8d29-96224fcfc997/dns/0.log"
Jul 29 16:42:30.631719 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:30.631684    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mszm6_8aac57b2-7e6a-45ec-8d29-96224fcfc997/kube-rbac-proxy/0.log"
Jul 29 16:42:30.756722 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:30.756692    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kgn8d_6fde166c-e320-4357-8289-4450abb2cc06/dns-node-resolver/0.log"
Jul 29 16:42:31.316982 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:31.316951    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5l247_5f5c78df-f246-43b5-a4c2-7704c6af4c1d/node-ca/0.log"
Jul 29 16:42:32.462767 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:32.462742    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86bbb74d46-5qf7q_2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf/kube-auth-proxy/0.log"
Jul 29 16:42:33.418608 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:33.418575    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6wtqz_4f7e28c6-0d23-445a-9b23-725d4dc65be1/serve-healthcheck-canary/0.log"
Jul 29 16:42:33.942482 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:33.942459    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2k9gn_41f96031-45cd-4059-9d82-e277dd98bb71/kube-rbac-proxy/0.log"
Jul 29 16:42:33.961061 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:33.961035    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2k9gn_41f96031-45cd-4059-9d82-e277dd98bb71/exporter/0.log"
Jul 29 16:42:33.980483 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:33.980459    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2k9gn_41f96031-45cd-4059-9d82-e277dd98bb71/extractor/0.log"
Jul 29 16:42:35.200617 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:35.200582    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kqvb7/perf-node-gather-daemonset-jswkp"
Jul 29 16:42:37.357158 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:37.357064    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75b794959f-zwzb7_7238c72e-3821-425c-8c55-e7e65ac8d3da/manager/0.log"
Jul 29 16:42:37.376428 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:37.376398    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75b794959f-zwzb7_7238c72e-3821-425c-8c55-e7e65ac8d3da/copy-manifests/0.log"
Jul 29 16:42:38.149197 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:38.149169    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86bbb74d46-5qf7q_2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf/kube-auth-proxy/0.log"
Jul 29 16:42:38.688193 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:38.688163    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7cb94b984b-c8gn9_0516333f-fe64-4021-b4c3-fbbbfd57c85b/manager/0.log"
Jul 29 16:42:38.735383 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:38.735358    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-hxgdj_7a49ed7a-ce4e-4ef6-9a89-86e6eaac5c15/openshift-lws-operator/0.log"
Jul 29 16:42:39.842654 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:39.842624    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86bbb74d46-5qf7q_2f91b2f6-1173-4fcd-95df-e41e5dc4a1bf/kube-auth-proxy/0.log"
Jul 29 16:42:44.855800 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:44.855723    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pbksq_eeeada33-be3e-45ce-b580-86be8ee80c48/kube-multus-additional-cni-plugins/0.log"
Jul 29 16:42:44.873790 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:44.873761    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pbksq_eeeada33-be3e-45ce-b580-86be8ee80c48/egress-router-binary-copy/0.log"
Jul 29 16:42:44.891804 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:44.891771    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pbksq_eeeada33-be3e-45ce-b580-86be8ee80c48/cni-plugins/0.log"
Jul 29 16:42:44.914615 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:44.914585    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pbksq_eeeada33-be3e-45ce-b580-86be8ee80c48/bond-cni-plugin/0.log"
Jul 29 16:42:44.934105 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:44.934076    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pbksq_eeeada33-be3e-45ce-b580-86be8ee80c48/routeoverride-cni/0.log"
Jul 29 16:42:44.951653 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:44.951618    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pbksq_eeeada33-be3e-45ce-b580-86be8ee80c48/whereabouts-cni-bincopy/0.log"
Jul 29 16:42:44.970531 ip-10-0-143-118 kubenswrapper[2560]: I0729 16:42:44.970501    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pbksq_eeeada33-be3e-45ce-b580-86be8ee80c48/whereabouts-cni/0.log"