Jul 24 18:42:11.245137 ip-10-0-141-213 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 24 18:42:11.245146 ip-10-0-141-213 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 24 18:42:11.245156 ip-10-0-141-213 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 24 18:42:11.245445 ip-10-0-141-213 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 24 18:42:21.257502 ip-10-0-141-213 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 24 18:42:21.257518 ip-10-0-141-213 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e97f7cf6b19a4bea9d678c9d6df10cab --
Jul 24 18:44:21.473655 ip-10-0-141-213 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 18:44:21.939538 ip-10-0-141-213 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 18:44:21.939538 ip-10-0-141-213 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 18:44:21.939538 ip-10-0-141-213 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 18:44:21.939538 ip-10-0-141-213 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 18:44:21.939538 ip-10-0-141-213 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 18:44:21.940552 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.940465    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 18:44:21.944810 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944794    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 18:44:21.944810 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944810    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944814    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944818    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944821    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944824    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944827    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944829    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944832    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944835    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944838    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944840    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944843    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944845    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944848    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944851    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944853    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944860    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944863    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944866    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944869    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 18:44:21.944872 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944871    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944874    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944877    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944880    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944883    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944885    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944888    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944890    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944893    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944896    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944898    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944900    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944903    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944905    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944909    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944913    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944915    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944923    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944926    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944929    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 18:44:21.945350 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944931    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944934    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944936    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944938    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944941    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944945    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944949    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944951    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944954    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944959    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944962    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944965    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944967    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944971    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944974    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944977    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944980    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944983    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944985    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 18:44:21.945857 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944988    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944990    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944993    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944995    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.944997    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945000    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945002    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945005    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945007    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945010    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945012    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945016    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945018    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945021    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945023    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945025    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945028    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945031    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945033    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945035    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 18:44:21.946311 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945038    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945040    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945043    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945045    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945049    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945051    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945055    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945058    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945061    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945063    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945066    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945068    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945071    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945073    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945076    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945078    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945081    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945083    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945086    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 18:44:21.946845 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945818    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945825    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945828    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945832    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945835    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945838    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945841    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945844    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945847    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945849    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945852    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945855    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945857    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945859    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945862    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945865    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945868    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945870    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945873    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945876    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 18:44:21.947301 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945879    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945882    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945885    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945887    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945890    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945894    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945898    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945901    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945904    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945906    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945909    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945912    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945914    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945916    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945924    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945927    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945929    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945932    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945935    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 18:44:21.947807 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945937    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945940    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945942    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945945    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945947    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945949    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945952    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945954    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945957    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945959    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945962    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945964    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945967    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945970    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945973    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945976    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945980    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945983    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945986    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 18:44:21.948265 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945989    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945991    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945994    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945996    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.945999    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946001    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946004    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946007    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946010    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946012    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946014    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946017    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946019    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946022    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946024    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946027    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946029    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946031    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946034    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946036    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 18:44:21.948778 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946039    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946041    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946043    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946046    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946048    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946051    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946054    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946056    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946059    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946061    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946064    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946066    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946068    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946071    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946073    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946076    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946078    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946081    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946083    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946086    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 18:44:21.949346 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946090    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946460    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946465    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946468    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946471    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946474    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946477    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946480    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946483    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946485    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946488    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946491    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946494    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946496    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946499    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946501    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946503    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946507    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946510    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946513    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 18:44:21.949852 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946515    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946518    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946521    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946523    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946526    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946529    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946531    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946534    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946536    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946539    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946541    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946544    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946546    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946549    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946554    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946557    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946560    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946563    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946567    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 18:44:21.950337 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946570    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946573    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946576    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946579    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946582    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946584    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946587    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946590    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946593    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946596    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946598    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946601    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946604    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946608    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946611    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946614    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946616    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946619    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946622    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946625    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 18:44:21.950814 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946627    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946630    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946633    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946636    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946639    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946642    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946646    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946649    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946652    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946655    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946657    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946659    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946662    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946665    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946670    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946673    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946675    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946678    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946680    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946682    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 18:44:21.951300 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946685    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946688    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946690    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946695    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946697    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946700    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946702    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946705    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946707    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946710    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946712    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946714    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946717    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946720    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946722    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946725    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946727    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946730    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946732    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946734    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 18:44:21.951809 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.946737    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946806    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946813    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946818    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946823    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946835    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946840    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946844    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946848    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946851    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946854    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946858    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946861    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946864    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946867    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946872    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946875    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946877    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946880    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946885    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946888    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946891    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946894    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946897    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 18:44:21.952281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946901    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946904    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946906    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946910    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946912    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946915    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946918    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946922    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946924    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946928    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946931    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946934    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946936    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946939    2563 flags.go:64] FLAG: --enable-server="true"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946944    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946949    2563 flags.go:64] FLAG: --event-burst="100"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946952    2563 flags.go:64] FLAG: --event-qps="50"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946955    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946958    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946961    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946964    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946967    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946970    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946974    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946978    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 18:44:21.952864 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946980    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946983    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946986    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946989    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946992    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946995    2563 flags.go:64] FLAG: --feature-gates=""
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.946999    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947002    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947004    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947008    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947010    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947014    2563 flags.go:64] FLAG: --help="false"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947016    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947020    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947023    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947025    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947029    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947032    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947035    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947038    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947040    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947044    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947048    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947051    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 18:44:21.953527 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947053    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947056    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947059    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947062    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947064    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947067    2563 flags.go:64] FLAG: --lock-file=""
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947071    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947074    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947076    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947081    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947084    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947087    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947089    2563 flags.go:64] FLAG: --logging-format="text"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947092    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947095    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947098    2563 flags.go:64] FLAG: --manifest-url=""
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947100    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947105    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947107    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947111    2563 flags.go:64] FLAG: --max-pods="110"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947114    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947117    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947120    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947122    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947125    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 18:44:21.954110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947128    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947131    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947137    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947140    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947145    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947148    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947151    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947156    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947159    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947162    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947178    2563 flags.go:64] FLAG: --port="10250"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947182    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947186    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-066c602210bbebfb1"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947189    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947193    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947196    2563 flags.go:64] FLAG: --register-node="true"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947199    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947202    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947205    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947208    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947211    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947217    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947220    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947223    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947225    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 18:44:21.955009 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947228    2563 flags.go:64] FLAG: --runonce="false"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947230    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947233    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947236    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947239    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947241    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947244    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947247    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947250    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947252    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947255    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947259    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947261    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947264    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947268    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947271    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947276    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947279    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947282    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947286    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947289    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947293    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947296    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947299    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947302    2563 flags.go:64] FLAG: --v="2"
Jul 24 18:44:21.955835 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947306    2563 flags.go:64] FLAG: --version="false"
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947310    2563 flags.go:64] FLAG: --vmodule=""
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947313    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.947317    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947459    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947465    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947469    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947472    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947475    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947478    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947481    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947483    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947486    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947488    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947491    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947493    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947496    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947498    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947501    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947504    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 18:44:21.956467 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947506    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947509    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947512    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947515    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947517    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947519    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947522    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947524    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947528    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947530    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947534    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947538    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947540    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947543    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947545    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947548    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947550    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947552    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947555    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947557    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 18:44:21.956958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947560    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947562    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947565    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947567    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947570    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947572    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947574    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947577    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947579    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947582    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947584    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947587    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947590    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947592    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947594    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947597    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947600    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947602    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947605    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947607    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 18:44:21.957451 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947611    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947613    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947616    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947618    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947621    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947623    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947625    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947628    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947630    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947633    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947635    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947638    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947640    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947642    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947645    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947647    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947651    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947655    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947657    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947660    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 18:44:21.957941 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947663    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947665    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947668    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947670    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947673    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947675    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947678    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947680    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947683    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947685    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947688    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947690    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947694    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947696    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947698    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947701    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947703    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947706    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947709    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 18:44:21.958466 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947711    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947713    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947715    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.947718    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.948635    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.956439    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.956454    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.956468    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.956474    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956534    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956538    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956542    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956545    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956548    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956551    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 18:44:21.958939 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956555    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956557    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956560    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956563    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956565    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956568    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956570    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956573    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956576    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956578    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956581    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956583    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956585    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956588    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956590    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956593    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956595    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956598    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956600    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956603    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 18:44:21.959307 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956606    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956609    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956611    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956615    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956620    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956623    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956626    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956629    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956632    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956634    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956637    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956639    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956642    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956644    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956647    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956649    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956651    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956654    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956656    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 18:44:21.959808 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956659    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956662    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956664    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956667    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956669    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956673    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956676    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956680    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956683    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956686    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956688    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956691    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956694    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956696    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956699    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956701    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956705    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956708    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956710    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 18:44:21.960272 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956712    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956715    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956718    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956720    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956722    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956725    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956727    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956730    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956732    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956734    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956737    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956739    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956742    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956744    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956747    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956749    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956752    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956754    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956757    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956759    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 18:44:21.960748 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956762    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 18:44:21.961238 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956764    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 18:44:21.961238 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956766    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 18:44:21.961238 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956769    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 18:44:21.961238 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956772    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 18:44:21.961238 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956774    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 18:44:21.961238 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956776    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 18:44:21.961238 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956779    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 18:44:21.961238 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956781    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 18:44:21.961238 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956784    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 18:44:21.961238 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956787    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 18:44:21.961238 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956789    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 18:44:21.961238 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956792    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 18:44:21.961238 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956795    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 18:44:21.961238 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956797    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.956802    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956933    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956938    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956941    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956944    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956947    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956949    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956952    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956954    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956957    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956960    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956962    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956965    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956968    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 18:44:21.961730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956970    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956973    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956975    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956978    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956980    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956984    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956987    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956990    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956993    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956995    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.956998    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957002    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957005    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957008    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957010    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957013    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957015    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957018    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957020    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 18:44:21.962104 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957023    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957025    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957028    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957030    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957032    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957035    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957037    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957040    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957042    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957045    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957047    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957049    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957052    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957055    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957057    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957060    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957062    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957064    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957067    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957069    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 18:44:21.962574 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957071    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957074    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957076    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957078    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957081    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957083    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957086    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957088    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957091    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957093    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957095    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957098    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957101    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957104    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957108    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957110    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957113    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957115    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957118    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 18:44:21.963050 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957120    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957123    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957125    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957128    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957130    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957132    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957135    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957138    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957140    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957142    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957145    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957147    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957150    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957152    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957154    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957157    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957160    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957162    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957164    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957170    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 18:44:21.963520 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957173    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 18:44:21.964003 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957175    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 18:44:21.964003 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957178    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 18:44:21.964003 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957180    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 18:44:21.964003 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957183    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 18:44:21.964003 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957186    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 18:44:21.964003 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957188    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 18:44:21.964003 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:21.957190    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 18:44:21.964003 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.957195    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 18:44:21.964003 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.957287    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 18:44:21.964003 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.959295    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 18:44:21.964003 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.960640    2563 server.go:1019] "Starting client certificate rotation"
Jul 24 18:44:21.964003 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.960746    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 18:44:21.964003 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.960777    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 18:44:21.964341 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.963222    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 18:44:21.964586 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.964570    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 18:44:21.977001 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.976977    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 24 18:44:21.979270 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.979255    2563 log.go:25] "Validated CRI v1 image API"
Jul 24 18:44:21.979997 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.979977    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 18:44:21.982363 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.982346    2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 85cd160b-b488-41fb-9d75-fe9328ac0100:/dev/nvme0n1p3 d8486dac-1179-43f3-9627-72d51423f5f3:/dev/nvme0n1p4]
Jul 24 18:44:21.982447 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.982363    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 18:44:21.986857 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.986839    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 18:44:21.990765 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.990647    2563 manager.go:216] Machine: {Timestamp:2026-07-24 18:44:21.989241347 +0000 UTC m=+0.380192815 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093049 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a7f1006b0295532e2bf1245e9a8aa SystemUUID:ec2a7f10-06b0-2955-32e2-bf1245e9a8aa BootID:e97f7cf6-b19a-4bea-9d67-8c9d6df10cab Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4f:e5:c4:c3:bd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4f:e5:c4:c3:bd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:b2:d4:09:ea:b6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 18:44:21.990765 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.990753    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 18:44:21.990900 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.990834    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 18:44:21.994007 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.993972    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 18:44:21.994172 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.994011    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-213.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 18:44:21.994768 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.994756    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 18:44:21.994839 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.994775    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 18:44:21.994839 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.994786    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 18:44:21.995226 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.995214    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 18:44:21.995973 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.995960    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 18:44:21.997636 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.997622    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 18:44:21.997943 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.997929    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 18:44:21.998006 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.997973    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 18:44:21.998006 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.997989    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 18:44:21.998100 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.998009    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 18:44:21.998100 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.998036    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 24 18:44:21.998100 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.998051    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 18:44:21.998484 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.998436    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 18:44:21.998484 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:21.998462    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 18:44:22.000738 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.000720    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 18:44:22.002405 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.002381    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 18:44:22.002454 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.002431    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 18:44:22.003806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.003794    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 18:44:22.003849 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.003816    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 18:44:22.003849 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.003826    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 18:44:22.003849 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.003833    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 18:44:22.003849 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.003847    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 18:44:22.003961 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.003856    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 18:44:22.003961 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.003866    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 18:44:22.003961 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.003875    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 18:44:22.003961 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.003882    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 18:44:22.003961 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.003887    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 18:44:22.003961 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.003903    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 18:44:22.003961 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.003909    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 18:44:22.004644 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.004622    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 18:44:22.004644 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.004631    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 18:44:22.007854 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.007840    2563 server.go:1268] "Started kubelet"
Jul 24 18:44:22.008058 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.008036    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:22.008058 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.007926    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 18:44:22.008179 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.008136    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 18:44:22.008230 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.008187    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 18:44:22.008425 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.008346    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 18:44:22.008570 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.008384    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 18:44:22.008771 ip-10-0-141-213 systemd[1]: Started Kubernetes Kubelet.
Jul 24 18:44:22.009712 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.009635    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 18:44:22.011300 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.011284    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 18:44:22.014076 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.012728    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f1f0b864  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.00779786 +0000 UTC m=+0.398749328,LastTimestamp:2026-07-24 18:44:22.00779786 +0000 UTC m=+0.398749328,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.014426 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.014387    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 18:44:22.016025 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.016011    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 18:44:22.016103 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.016030    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 18:44:22.016103 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.016077    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:22.016681 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.016666    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 18:44:22.016681 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.016672    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 18:44:22.016863 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.016692    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 18:44:22.016863 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.016770    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 18:44:22.016863 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.016778    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 18:44:22.016863 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.016806    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 18:44:22.016863 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.016827    2563 factory.go:55] Registering systemd factory
Jul 24 18:44:22.016863 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.016835    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 24 18:44:22.017223 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.017099    2563 factory.go:153] Registering CRI-O factory
Jul 24 18:44:22.017223 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.017113    2563 factory.go:223] Registration of the crio container factory successfully
Jul 24 18:44:22.017223 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.017138    2563 factory.go:103] Registering Raw factory
Jul 24 18:44:22.017223 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.017152    2563 manager.go:1196] Started watching for new ooms in manager
Jul 24 18:44:22.017589 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.017574    2563 manager.go:349] Starting recovery of all containers
Jul 24 18:44:22.018353 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.018331    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 18:44:22.023494 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.023315    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mdfdk"
Jul 24 18:44:22.029254 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.029236    2563 manager.go:354] Recovery completed
Jul 24 18:44:22.033753 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.033740    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:22.034052 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.034034    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 18:44:22.036292 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.036277    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:22.036370 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.036306    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:22.036370 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.036319    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:22.036769 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.036752    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 18:44:22.036769 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.036764    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 18:44:22.036957 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.036781    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 18:44:22.036957 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.036783    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:22.037448 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.037431    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:22.037524 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.037461    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:22.037524 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.037475    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:22.038387 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.038375    2563 policy_none.go:49] "None policy: Start"
Jul 24 18:44:22.038387 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.038409    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 18:44:22.038490 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.038422    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 18:44:22.039370 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.039359    2563 policy_none.go:47] "Start"
Jul 24 18:44:22.045715 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.045641    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a37fed  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036291565 +0000 UTC m=+0.427243033,LastTimestamp:2026-07-24 18:44:22.036291565 +0000 UTC m=+0.427243033,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.050226 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.050162    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3d554  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036313428 +0000 UTC m=+0.427264895,LastTimestamp:2026-07-24 18:44:22.036313428 +0000 UTC m=+0.427264895,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.058228 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.058165    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3fd41  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036323649 +0000 UTC m=+0.427275116,LastTimestamp:2026-07-24 18:44:22.036323649 +0000 UTC m=+0.427275116,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.068159 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.068094    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a37fed\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a37fed  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036291565 +0000 UTC m=+0.427243033,LastTimestamp:2026-07-24 18:44:22.037446529 +0000 UTC m=+0.428397999,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.072973 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.072957    2563 manager.go:337] "Starting Device Plugin manager"
Jul 24 18:44:22.073070 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.073007    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 18:44:22.073070 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.073022    2563 server.go:87] "Starting device plugin registration server"
Jul 24 18:44:22.073287 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.073274    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 18:44:22.073343 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.073289    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 18:44:22.073418 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.073384    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 18:44:22.073487 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.073476    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 18:44:22.073487 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.073487    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 18:44:22.074181 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.074163    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 18:44:22.074250 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.074208    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:22.077613 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.077556    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a3d554\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3d554  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036313428 +0000 UTC m=+0.427264895,LastTimestamp:2026-07-24 18:44:22.037468049 +0000 UTC m=+0.428419520,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.085997 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.085934    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a3fd41\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3fd41  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036323649 +0000 UTC m=+0.427275116,LastTimestamp:2026-07-24 18:44:22.037479135 +0000 UTC m=+0.428430605,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.097459 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.097384    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f5f695de  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.075291102 +0000 UTC m=+0.466242563,LastTimestamp:2026-07-24 18:44:22.075291102 +0000 UTC m=+0.466242563,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.173901 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.173854    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:22.175161 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.175138    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:22.175272 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.175174    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:22.175272 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.175191    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:22.175272 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.175224    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.191106 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.190950    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a37fed\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a37fed  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036291565 +0000 UTC m=+0.427243033,LastTimestamp:2026-07-24 18:44:22.175156188 +0000 UTC m=+0.566107660,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.195586 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.195564    2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.196036 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.195969    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a3d554\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3d554  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036313428 +0000 UTC m=+0.427264895,LastTimestamp:2026-07-24 18:44:22.175180472 +0000 UTC m=+0.566131942,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.196138 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.196107    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 18:44:22.197709 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.197690    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 18:44:22.197771 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.197738    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 18:44:22.197771 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.197762    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 18:44:22.197833 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.197797    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 18:44:22.199243 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.199174    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a3fd41\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3fd41  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036323649 +0000 UTC m=+0.427275116,LastTimestamp:2026-07-24 18:44:22.175196518 +0000 UTC m=+0.566147987,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.209077 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.209059    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 24 18:44:22.229132 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.229089    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms"
Jul 24 18:44:22.298359 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.298314    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal"]
Jul 24 18:44:22.298477 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.298428    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:22.300019 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.300002    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:22.300084 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.300034    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:22.300084 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.300044    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:22.301311 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.301292    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:22.301449 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.301434    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.301499 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.301464    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:22.302023 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.302005    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:22.302106 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.302035    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:22.302106 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.302048    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:22.302106 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.302007    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:22.302214 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.302115    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:22.302214 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.302128    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:22.303208 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.303193    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.303259 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.303217    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:22.303884 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.303867    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:22.303991 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.303892    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:22.303991 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.303904    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:22.312130 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.312062    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a37fed\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a37fed  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036291565 +0000 UTC m=+0.427243033,LastTimestamp:2026-07-24 18:44:22.300019361 +0000 UTC m=+0.690970829,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.318811 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.318795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49c209b3e6e7d8404ad2a977c0bb87c9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal\" (UID: \"49c209b3e6e7d8404ad2a977c0bb87c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.318878 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.318818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a2ba8c9c359a1f1e65affdf9bc68bd46-config\") pod \"kube-apiserver-proxy-ip-10-0-141-213.ec2.internal\" (UID: \"a2ba8c9c359a1f1e65affdf9bc68bd46\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.318878 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.318843    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49c209b3e6e7d8404ad2a977c0bb87c9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal\" (UID: \"49c209b3e6e7d8404ad2a977c0bb87c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.320856 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.320798    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a3d554\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3d554  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036313428 +0000 UTC m=+0.427264895,LastTimestamp:2026-07-24 18:44:22.300038538 +0000 UTC m=+0.690990006,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.323947 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.323932    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-213.ec2.internal\" not found" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.328386 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.328368    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-213.ec2.internal\" not found" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.330473 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.330415    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a3fd41\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3fd41  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036323649 +0000 UTC m=+0.427275116,LastTimestamp:2026-07-24 18:44:22.300047702 +0000 UTC m=+0.690999170,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.341184 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.341116    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a37fed\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a37fed  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036291565 +0000 UTC m=+0.427243033,LastTimestamp:2026-07-24 18:44:22.302023298 +0000 UTC m=+0.692974766,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.353159 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.353101    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a3d554\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3d554  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036313428 +0000 UTC m=+0.427264895,LastTimestamp:2026-07-24 18:44:22.302042119 +0000 UTC m=+0.692993587,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.362210 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.362153    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a3fd41\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3fd41  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036323649 +0000 UTC m=+0.427275116,LastTimestamp:2026-07-24 18:44:22.302052076 +0000 UTC m=+0.693003544,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.377545 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.377488    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a37fed\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a37fed  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036291565 +0000 UTC m=+0.427243033,LastTimestamp:2026-07-24 18:44:22.302101722 +0000 UTC m=+0.693053196,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.395989 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.395965    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:22.396871 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.396833    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:22.396967 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.396886    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:22.396967 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.396902    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:22.396967 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.396936    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.397082 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.397005    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a3d554\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3d554  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036313428 +0000 UTC m=+0.427264895,LastTimestamp:2026-07-24 18:44:22.302123322 +0000 UTC m=+0.693074790,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.407232 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.407149    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a3fd41\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3fd41  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036323649 +0000 UTC m=+0.427275116,LastTimestamp:2026-07-24 18:44:22.302132517 +0000 UTC m=+0.693083986,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.410424 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.410341    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a37fed\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a37fed  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036291565 +0000 UTC m=+0.427243033,LastTimestamp:2026-07-24 18:44:22.303881921 +0000 UTC m=+0.694833388,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.410552 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.410530    2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.419088 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.419066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49c209b3e6e7d8404ad2a977c0bb87c9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal\" (UID: \"49c209b3e6e7d8404ad2a977c0bb87c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.419148 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.419091    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49c209b3e6e7d8404ad2a977c0bb87c9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal\" (UID: \"49c209b3e6e7d8404ad2a977c0bb87c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.419148 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.419106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a2ba8c9c359a1f1e65affdf9bc68bd46-config\") pod \"kube-apiserver-proxy-ip-10-0-141-213.ec2.internal\" (UID: \"a2ba8c9c359a1f1e65affdf9bc68bd46\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.419148 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.419141    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a2ba8c9c359a1f1e65affdf9bc68bd46-config\") pod \"kube-apiserver-proxy-ip-10-0-141-213.ec2.internal\" (UID: \"a2ba8c9c359a1f1e65affdf9bc68bd46\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.419257 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.419160    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49c209b3e6e7d8404ad2a977c0bb87c9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal\" (UID: \"49c209b3e6e7d8404ad2a977c0bb87c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.419257 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.419174    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49c209b3e6e7d8404ad2a977c0bb87c9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal\" (UID: \"49c209b3e6e7d8404ad2a977c0bb87c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.425724 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.425664    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a3d554\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3d554  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036313428 +0000 UTC m=+0.427264895,LastTimestamp:2026-07-24 18:44:22.303897007 +0000 UTC m=+0.694848475,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.432159 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.432100    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a3fd41\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3fd41  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036323649 +0000 UTC m=+0.427275116,LastTimestamp:2026-07-24 18:44:22.303907678 +0000 UTC m=+0.694859146,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.436777 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.436716    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a37fed\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a37fed  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036291565 +0000 UTC m=+0.427243033,LastTimestamp:2026-07-24 18:44:22.396870038 +0000 UTC m=+0.787821506,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.444465 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.444367    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-213.ec2.internal.18c54e10f3a3d554\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18c54e10f3a3d554  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-213.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:22.036313428 +0000 UTC m=+0.427264895,LastTimestamp:2026-07-24 18:44:22.396893349 +0000 UTC m=+0.787845004,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:22.626508 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.626468    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.632325 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.632309    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.640883 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.640857    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="800ms"
Jul 24 18:44:22.811575 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.811551    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:22.818288 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.818260    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:22.818427 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.818296    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:22.818427 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.818314    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:22.818427 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:22.818356    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:22.837953 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:22.837924    2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:23.018351 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:23.018317    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:23.063238 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:23.063169    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 18:44:23.094833 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:23.094687    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:23.095943 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:23.095923    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:23.096025 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:23.095959    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:23.096025 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:23.095977    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:23.199032 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:23.198992    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ba8c9c359a1f1e65affdf9bc68bd46.slice/crio-9839b879c983b0c445a95578e6589d46280b160495e09d285f4191c9b7cc600d WatchSource:0}: Error finding container 9839b879c983b0c445a95578e6589d46280b160495e09d285f4191c9b7cc600d: Status 404 returned error can't find the container with id 9839b879c983b0c445a95578e6589d46280b160495e09d285f4191c9b7cc600d
Jul 24 18:44:23.203250 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:23.203232    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 18:44:23.212045 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:23.211967    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal.18c54e113934dea5  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal,UID:49c209b3e6e7d8404ad2a977c0bb87c9,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\",Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:23.203446437 +0000 UTC m=+1.594397901,LastTimestamp:2026-07-24 18:44:23.203446437 +0000 UTC m=+1.594397901,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:23.222882 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:23.222809    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-141-213.ec2.internal.18c54e1139352928  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-141-213.ec2.internal,UID:a2ba8c9c359a1f1e65affdf9bc68bd46,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\",Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:23.203465512 +0000 UTC m=+1.594416972,LastTimestamp:2026-07-24 18:44:23.203465512 +0000 UTC m=+1.594416972,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:23.453738 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:23.453638    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 18:44:23.453738 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:23.453667    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="1.6s"
Jul 24 18:44:23.599795 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:23.599758    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 24 18:44:23.617053 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:23.617010    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 18:44:23.638580 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:23.638547    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:23.639522 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:23.639500    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:23.639658 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:23.639540    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:23.639658 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:23.639551    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:23.639658 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:23.639579    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:23.660107 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:23.660080    2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:24.020112 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:24.020076    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:24.202776 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:24.202707    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" event={"ID":"49c209b3e6e7d8404ad2a977c0bb87c9","Type":"ContainerStarted","Data":"4700a9d706a6b52dc763af2f6f7e79b91e528d480e522186ec9f93221ed8cdea"}
Jul 24 18:44:24.204302 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:24.204272    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal" event={"ID":"a2ba8c9c359a1f1e65affdf9bc68bd46","Type":"ContainerStarted","Data":"9839b879c983b0c445a95578e6589d46280b160495e09d285f4191c9b7cc600d"}
Jul 24 18:44:24.975374 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:24.975346    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 18:44:25.018307 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:25.018278    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:25.065236 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:25.065203    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="3.2s"
Jul 24 18:44:25.132772 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:25.132679    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal.18c54e11aba82aa8  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal,UID:49c209b3e6e7d8404ad2a977c0bb87c9,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\" in 1.92s (1.92s including waiting). Image size: 278280461 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:25.12360516 +0000 UTC m=+3.514556617,LastTimestamp:2026-07-24 18:44:25.12360516 +0000 UTC m=+3.514556617,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:25.156584 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:25.155229    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-141-213.ec2.internal.18c54e11acd669b4  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-141-213.ec2.internal,UID:a2ba8c9c359a1f1e65affdf9bc68bd46,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\" in 1.939s (1.939s including waiting). Image size: 317186379 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:25.143413172 +0000 UTC m=+3.534364628,LastTimestamp:2026-07-24 18:44:25.143413172 +0000 UTC m=+3.534364628,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:25.217170 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:25.216965    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-141-213.ec2.internal.18c54e11b0784e4a  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-141-213.ec2.internal,UID:a2ba8c9c359a1f1e65affdf9bc68bd46,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Created,Message:Created container: haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:25.204354634 +0000 UTC m=+3.595306111,LastTimestamp:2026-07-24 18:44:25.204354634 +0000 UTC m=+3.595306111,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:25.227287 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:25.227195    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-141-213.ec2.internal.18c54e11b1209466  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-141-213.ec2.internal,UID:a2ba8c9c359a1f1e65affdf9bc68bd46,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Started,Message:Started container haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:25.21538263 +0000 UTC m=+3.606334107,LastTimestamp:2026-07-24 18:44:25.21538263 +0000 UTC m=+3.606334107,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:25.260559 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:25.260526    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:25.261723 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:25.261701    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:25.261813 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:25.261739    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:25.261813 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:25.261752    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:25.261813 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:25.261783    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:25.282669 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:25.282643    2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:25.340931 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:25.340845    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal.18c54e11b80afa92  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal,UID:49c209b3e6e7d8404ad2a977c0bb87c9,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:25.331407506 +0000 UTC m=+3.722358984,LastTimestamp:2026-07-24 18:44:25.331407506 +0000 UTC m=+3.722358984,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:25.350078 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:25.350058    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:25.350956 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:25.350935    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:25.351042 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:25.350976    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:25.351042 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:25.350993    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:25.351500 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:25.351398    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal.18c54e11b891f238  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal,UID:49c209b3e6e7d8404ad2a977c0bb87c9,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:25.340252728 +0000 UTC m=+3.731204202,LastTimestamp:2026-07-24 18:44:25.340252728 +0000 UTC m=+3.731204202,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:26.019276 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:26.019248    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:26.136738 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:26.136696    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 24 18:44:26.215017 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:26.214983    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal" event={"ID":"a2ba8c9c359a1f1e65affdf9bc68bd46","Type":"ContainerStarted","Data":"f4a367cc6ae6fde61eb2c42fac1d352c63b93608aaad37ae14348b6afaf9f4c5"}
Jul 24 18:44:26.215017 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:26.215011    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:26.215879 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:26.215859    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:26.215994 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:26.215888    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:26.215994 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:26.215899    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:26.216104 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:26.216080    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-213.ec2.internal\" not found" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:26.216555 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:26.216533    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" event={"ID":"49c209b3e6e7d8404ad2a977c0bb87c9","Type":"ContainerStarted","Data":"97c4151a1126ee7d80b05bf08c5bd72c02e694631cb8f93e9448c14b5d7e78f2"}
Jul 24 18:44:26.216657 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:26.216591    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:26.217304 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:26.217286    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:26.217383 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:26.217317    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:26.217383 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:26.217328    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:26.217510 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:26.217497    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-213.ec2.internal\" not found" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:26.265574 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:26.265542    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 18:44:26.828690 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:26.828651    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 18:44:27.019333 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:27.019297    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:27.217621 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:27.217546    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:27.218019 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:27.217546    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:27.218449 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:27.218431    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:27.218449 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:27.218419    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:27.218630 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:27.218464    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:27.218630 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:27.218475    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:27.218630 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:27.218491    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:27.218630 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:27.218479    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:27.218825 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:27.218758    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-213.ec2.internal\" not found" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:27.218884 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:27.218868    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-213.ec2.internal\" not found" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:28.021346 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:28.021309    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:28.279056 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:28.278965    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="6.4s"
Jul 24 18:44:28.483220 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:28.483174    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:28.485357 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:28.485334    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:28.485504 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:28.485370    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:28.485504 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:28.485381    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:28.485504 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:28.485428    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:28.509270 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:28.509241    2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:28.659493 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:28.659420    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 18:44:29.019061 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:29.019007    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:29.633254 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:29.633214    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 18:44:29.985193 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:29.985150    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:29.986214 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:29.986196    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:29.986306 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:29.986228    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:29.986306 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:29.986238    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:30.019349 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:30.019319    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:30.808708 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:30.808671    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 18:44:30.867066 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:30.867027    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 24 18:44:31.019115 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:31.019080    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:32.017079 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:32.017039    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:32.075330 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:32.075278    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:33.018537 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:33.018503    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:34.019070 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:34.019030    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:34.688551 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:34.688516    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 18:44:34.909908 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:34.909855    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:34.910823 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:34.910804    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:34.910918 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:34.910843    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:34.910918 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:34.910861    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:34.910918 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:34.910902    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:34.928328 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:34.928300    2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:35.019106 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:35.019032    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:35.762567 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:35.762532    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 18:44:36.020463 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:36.020407    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:37.017181 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:37.017146    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:37.236584 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:37.236556    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal_49c209b3e6e7d8404ad2a977c0bb87c9/setup/0.log"
Jul 24 18:44:37.236942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:37.236591    2563 generic.go:356] "Generic (PLEG): container finished" podID="49c209b3e6e7d8404ad2a977c0bb87c9" containerID="97c4151a1126ee7d80b05bf08c5bd72c02e694631cb8f93e9448c14b5d7e78f2" exitCode=1
Jul 24 18:44:37.236942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:37.236623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" event={"ID":"49c209b3e6e7d8404ad2a977c0bb87c9","Type":"ContainerDied","Data":"97c4151a1126ee7d80b05bf08c5bd72c02e694631cb8f93e9448c14b5d7e78f2"}
Jul 24 18:44:37.236942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:37.236746    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:37.237523 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:37.237507    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:37.237594 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:37.237539    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:37.237594 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:37.237548    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:37.237740 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:37.237728    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-213.ec2.internal\" not found" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:37.250365 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:37.250257    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal.18c54e147de482e8  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal,UID:49c209b3e6e7d8404ad2a977c0bb87c9,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:37.240709864 +0000 UTC m=+15.631661319,LastTimestamp:2026-07-24 18:44:37.240709864 +0000 UTC m=+15.631661319,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:37.323307 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:37.323233    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal.18c54e11b80afa92\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal.18c54e11b80afa92  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal,UID:49c209b3e6e7d8404ad2a977c0bb87c9,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:25.331407506 +0000 UTC m=+3.722358984,LastTimestamp:2026-07-24 18:44:37.311203507 +0000 UTC m=+15.702154977,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:37.331434 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:37.331320    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal.18c54e11b891f238\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal.18c54e11b891f238  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal,UID:49c209b3e6e7d8404ad2a977c0bb87c9,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-24 18:44:25.340252728 +0000 UTC m=+3.731204202,LastTimestamp:2026-07-24 18:44:37.320522452 +0000 UTC m=+15.711473926,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}"
Jul 24 18:44:37.730070 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:37.729987    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 18:44:38.018938 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:38.018871    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:38.086272 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:38.086251    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:38.087219 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:38.087201    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:38.087307 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:38.087232    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:38.087307 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:38.087242    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:38.239466 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:38.239437    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal_49c209b3e6e7d8404ad2a977c0bb87c9/setup/0.log"
Jul 24 18:44:38.239802 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:38.239485    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" event={"ID":"49c209b3e6e7d8404ad2a977c0bb87c9","Type":"ContainerStarted","Data":"5181529562ed83993a121ee557d5a60fb4454bd55b60c04c7a28ceaf2d3f8990"}
Jul 24 18:44:38.239802 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:38.239595    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:38.240347 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:38.240330    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:38.240457 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:38.240360    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:38.240457 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:38.240373    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:38.240591 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:38.240578    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-213.ec2.internal\" not found" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:39.017550 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:39.017521    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:40.018063 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:40.018035    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:41.016664 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:41.016637    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:41.449960 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:41.449871    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 18:44:41.698738 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:41.698696    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 18:44:41.928726 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:41.928649    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:41.929632 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:41.929609    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:41.929746 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:41.929646    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:41.929746 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:41.929662    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:41.929746 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:41.929690    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:41.946075 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:41.946047    2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:42.018021 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:42.017997    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:42.076505 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:42.076474    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:42.908789 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:42.908751    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 24 18:44:43.017119 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:43.017085    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:44.017969 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:44.017930    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:45.019244 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:45.019210    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 18:44:45.780345 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:45.780309    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mdfdk"
Jul 24 18:44:45.960074 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:45.960035    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 18:44:46.032572 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:46.032502    2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:46.053680 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:46.053658    2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:46.114100 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:46.114075    2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:46.394566 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:46.394540    2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:46.394566 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:46.394564    2563 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:46.430604 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:46.430586    2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:46.448825 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:46.448810    2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:46.508940 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:46.508920    2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:46.770546 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:46.770480    2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:46.770546 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:46.770501    2563 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:46.782649 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:46.782615    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 18:39:45 +0000 UTC" deadline="2028-04-20 22:40:46.762852032 +0000 UTC"
Jul 24 18:44:46.782649 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:46.782648    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15267h55m59.980207256s"
Jul 24 18:44:47.017350 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:47.017327    2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:47.033726 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:47.033687    2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:47.092851 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:47.092829    2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:47.355247 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:47.355222    2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:47.355247 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:47.355247    2563 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-141-213.ec2.internal" not found
Jul 24 18:44:48.705065 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:48.704979    2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-213.ec2.internal\" not found" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:48.946979 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:48.946930    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:48.948049 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:48.948029    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:48.948159 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:48.948065    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:48.948159 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:48.948081    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:48.948159 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:48.948117    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:48.955998 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:48.955953    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:48.955998 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:48.955977    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-213.ec2.internal\": node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:48.972082 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:48.972058    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:49.073052 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:49.073011    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:49.173640 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:49.173601    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:49.257785 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:49.257725    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal_49c209b3e6e7d8404ad2a977c0bb87c9/setup/1.log"
Jul 24 18:44:49.258127 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:49.258113    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal_49c209b3e6e7d8404ad2a977c0bb87c9/setup/0.log"
Jul 24 18:44:49.258163 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:49.258146    2563 generic.go:356] "Generic (PLEG): container finished" podID="49c209b3e6e7d8404ad2a977c0bb87c9" containerID="5181529562ed83993a121ee557d5a60fb4454bd55b60c04c7a28ceaf2d3f8990" exitCode=1
Jul 24 18:44:49.258195 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:49.258173    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" event={"ID":"49c209b3e6e7d8404ad2a977c0bb87c9","Type":"ContainerDied","Data":"5181529562ed83993a121ee557d5a60fb4454bd55b60c04c7a28ceaf2d3f8990"}
Jul 24 18:44:49.258227 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:49.258211    2563 scope.go:117] "RemoveContainer" containerID="97c4151a1126ee7d80b05bf08c5bd72c02e694631cb8f93e9448c14b5d7e78f2"
Jul 24 18:44:49.258304 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:49.258292    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:49.259884 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:49.259868    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:49.259967 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:49.259904    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:49.259967 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:49.259920    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:49.260158 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:49.260144    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-213.ec2.internal\" not found" node="ip-10-0-141-213.ec2.internal"
Jul 24 18:44:49.260251 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:49.260205    2563 scope.go:117] "RemoveContainer" containerID="97c4151a1126ee7d80b05bf08c5bd72c02e694631cb8f93e9448c14b5d7e78f2"
Jul 24 18:44:49.268565 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:49.268536    2563 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal_openshift-machine-config-operator_49c209b3e6e7d8404ad2a977c0bb87c9_0 in pod sandbox 4700a9d706a6b52dc763af2f6f7e79b91e528d480e522186ec9f93221ed8cdea from index: no such id: '97c4151a1126ee7d80b05bf08c5bd72c02e694631cb8f93e9448c14b5d7e78f2'" containerID="97c4151a1126ee7d80b05bf08c5bd72c02e694631cb8f93e9448c14b5d7e78f2"
Jul 24 18:44:49.268665 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:49.268590    2563 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal_openshift-machine-config-operator_49c209b3e6e7d8404ad2a977c0bb87c9_0 in pod sandbox 4700a9d706a6b52dc763af2f6f7e79b91e528d480e522186ec9f93221ed8cdea from index: no such id: '97c4151a1126ee7d80b05bf08c5bd72c02e694631cb8f93e9448c14b5d7e78f2'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal_openshift-machine-config-operator(49c209b3e6e7d8404ad2a977c0bb87c9)\"" logger="UnhandledError"
Jul 24 18:44:49.268777 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:49.268755    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal_openshift-machine-config-operator(49c209b3e6e7d8404ad2a977c0bb87c9)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" podUID="49c209b3e6e7d8404ad2a977c0bb87c9"
Jul 24 18:44:49.274684 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:49.274668    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:49.375794 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:49.375753    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:49.476398 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:49.476356    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:49.576993 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:49.576970    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:49.677468 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:49.677435    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:49.777949 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:49.777918    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:49.878480 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:49.878383    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:49.979555 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:49.979518    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:50.080425 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:50.080362    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:50.181014 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:50.180924    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:50.260977 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:50.260955    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal_49c209b3e6e7d8404ad2a977c0bb87c9/setup/1.log"
Jul 24 18:44:50.281998 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:50.281979    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:50.382476 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:50.382436    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:50.483123 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:50.483050    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:50.583726 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:50.583692    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:50.684119 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:50.684076    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:50.785066 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:50.785004    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:50.885490 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:50.885450    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:50.986044 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:50.986007    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:51.086954 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:51.086936    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:51.187618 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:51.187584    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:51.288010 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:51.287984    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:51.388935 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:51.388861    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:51.489452 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:51.489426    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:51.590001 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:51.589972    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:51.690520 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:51.690432    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:51.791294 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:51.791269    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:51.891757 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:51.891722    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:51.937641 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:51.937615    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 18:44:51.992308 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:51.992249    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:52.076838 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:52.076791    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:52.093126 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:52.093102    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:52.193579 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:52.193552    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:52.293844 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:52.293749    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:52.394433 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:52.394373    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:52.494542 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:52.494513    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:52.595101 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:52.595080    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:52.695838 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:52.695803    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:52.796267 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:52.796237    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:52.897167 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:52.897096    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:52.997594 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:52.997564    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:53.098691 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:53.098659    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:53.199387 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:53.199298    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:53.300136 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:53.300087    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:53.400517 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:53.400476    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:53.501519 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:53.501455    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:53.602154 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:53.602128    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:53.702253 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:53.702210    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:53.802688 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:53.802619    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:53.903058 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:53.903018    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:54.003643 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:54.003615    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:54.104586 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:54.104556    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:54.205027 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:54.204994    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:54.305362 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:54.305326    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:54.343740 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:54.343702    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 18:44:54.344837 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:54.344824    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 18:44:54.344909 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:54.344853    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 18:44:54.344909 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:54.344863    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID"
Jul 24 18:44:54.405829 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:54.405743    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:54.506364 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:54.506343    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:54.606818 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:54.606790    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:54.707214 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:54.707149    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:54.807665 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:54.807629    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:54.908518 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:54.908478    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:55.009376 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:55.009302    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:55.109526 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:55.109500    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:55.210479 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:55.210446    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:55.311318 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:55.311289    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:55.411770 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:55.411721    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:55.512371 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:55.512331    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:55.612793 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:55.612725    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found"
Jul 24 18:44:55.668188 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:55.668161    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 18:44:55.716418 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:55.716383    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:55.732474 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:55.732455    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 18:44:55.733932 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:55.733919    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal"
Jul 24 18:44:55.747981 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:55.747962    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 18:44:56.014813 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.014738    2563 apiserver.go:52] "Watching apiserver"
Jul 24 18:44:56.023571 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.023547    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 18:44:56.023934 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.023914    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb","openshift-cluster-node-tuning-operator/tuned-qm2vs","openshift-dns/node-resolver-qmjbp","openshift-multus/multus-additional-cni-plugins-tpk4s","openshift-multus/multus-fmsn5","openshift-network-diagnostics/network-check-target-psbp5","kube-system/global-pull-secret-syncer-jwsd9","kube-system/konnectivity-agent-9k9h7","kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal","openshift-image-registry/node-ca-n89pr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal","openshift-multus/network-metrics-daemon-zgg7r","openshift-network-operator/iptables-alerter-dtpsd","openshift-ovn-kubernetes/ovnkube-node-889p5"]
Jul 24 18:44:56.025875 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.025860    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qmjbp"
Jul 24 18:44:56.027314 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.027291    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9k9h7"
Jul 24 18:44:56.028330 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.028308    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 18:44:56.028549 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.028530    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 18:44:56.028549 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.028540    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4w9p8\""
Jul 24 18:44:56.028687 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.028577    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:44:56.028687 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.028653    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:44:56.029839 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.029817    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.031000 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.030980    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 18:44:56.031000 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.030996    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 18:44:56.031146 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.031042    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-stdbq\""
Jul 24 18:44:56.031436 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.031411    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.031776 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.031761    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zf8t7\""
Jul 24 18:44:56.031939 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.031923    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 18:44:56.032256 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.032239    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 18:44:56.032327 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.032258    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 18:44:56.032327 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.032258    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 18:44:56.032327 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.032239    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 18:44:56.032813 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.032525    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:44:56.032813 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.032590    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:44:56.033996 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.033980    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 18:44:56.034255 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.034235    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.034373 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.034358    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-77sjv\""
Jul 24 18:44:56.034508 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.034493    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 18:44:56.035409 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.035377    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.036621 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.036599    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 18:44:56.036722 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.036635    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 18:44:56.036825 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.036806    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 18:44:56.036888 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.036847    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lsnbz\""
Jul 24 18:44:56.037156 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.037140    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n89pr"
Jul 24 18:44:56.037578 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.037548    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 18:44:56.037578 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.037569    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t7pf4\""
Jul 24 18:44:56.037804 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.037788    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:44:56.037875 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.037862    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 18:44:56.037919 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.037855    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:44:56.038908 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.038889    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dtpsd"
Jul 24 18:44:56.040536 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.040109    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 18:44:56.040853 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.040827    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 18:44:56.040941 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.040862    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 18:44:56.041078 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.041062    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-drt4g\""
Jul 24 18:44:56.042073 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.041661    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 18:44:56.042209 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.042180    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.042654 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.042629    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal_openshift-machine-config-operator(49c209b3e6e7d8404ad2a977c0bb87c9)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" podUID="49c209b3e6e7d8404ad2a977c0bb87c9"
Jul 24 18:44:56.043637 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.043618    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 18:44:56.043914 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.043892    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 18:44:56.044244 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.044226    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-khfl8\""
Jul 24 18:44:56.046309 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.046295    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 18:44:56.046309 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.046306    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 18:44:56.046556 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.046535    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 18:44:56.046556 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.046550    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 18:44:56.046713 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.046570    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 18:44:56.046713 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.046536    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 18:44:56.047360 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.047343    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dcjck\""
Jul 24 18:44:56.054587 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.054569    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 18:44:56.074760 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.074743    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hzjh7"
Jul 24 18:44:56.083950 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.083933    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hzjh7"
Jul 24 18:44:56.117663 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.117638    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 18:44:56.145124 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.145072    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal" podStartSLOduration=1.145060133 podStartE2EDuration="1.145060133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:44:56.145006038 +0000 UTC m=+34.535957514" watchObservedRunningTime="2026-07-24 18:44:56.145060133 +0000 UTC m=+34.536011587"
Jul 24 18:44:56.214980 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.214950    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5eda199b-9a46-4bdd-919e-dbacd4631428-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.214980 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.214980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-var-lib-cni-bin\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.215217 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.214998    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-sysctl-d\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.215217 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215012    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-lib-modules\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.215217 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8f329d8-6209-4a6c-8004-60a1514c92ec-serviceca\") pod \"node-ca-n89pr\" (UID: \"a8f329d8-6209-4a6c-8004-60a1514c92ec\") " pod="openshift-image-registry/node-ca-n89pr"
Jul 24 18:44:56.215217 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215063    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-node-log\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.215217 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215104    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-cnibin\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.215217 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215137    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-slash\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.215217 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215166    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-run-systemd\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.215217 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215189    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-cni-netd\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.215477 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215220    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-etc-selinux\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.215477 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215250    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfz9h\" (UniqueName: \"kubernetes.io/projected/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-kube-api-access-bfz9h\") pod \"network-metrics-daemon-zgg7r\" (UID: \"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19\") " pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:44:56.215477 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215282    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe6a8da4-b859-441c-9001-7b14d9c8f065-tmp-dir\") pod \"node-resolver-qmjbp\" (UID: \"fe6a8da4-b859-441c-9001-7b14d9c8f065\") " pod="openshift-dns/node-resolver-qmjbp"
Jul 24 18:44:56.215477 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215300    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-var-lib-cni-multus\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.215477 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-run\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.215477 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215353    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-run-ovn\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.215477 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215376    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.215477 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fe6a8da4-b859-441c-9001-7b14d9c8f065-hosts-file\") pod \"node-resolver-qmjbp\" (UID: \"fe6a8da4-b859-441c-9001-7b14d9c8f065\") " pod="openshift-dns/node-resolver-qmjbp"
Jul 24 18:44:56.215477 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215444    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-os-release\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.215477 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215463    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-etc-kubernetes\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215500    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76e6dbde-7988-46ac-91e5-872bcc2ee08c-tmp\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215517    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8f329d8-6209-4a6c-8004-60a1514c92ec-host\") pod \"node-ca-n89pr\" (UID: \"a8f329d8-6209-4a6c-8004-60a1514c92ec\") " pod="openshift-image-registry/node-ca-n89pr"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215533    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-run-openvswitch\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/33101463-3e59-49a7-91d4-947558c4b6c7-iptables-alerter-script\") pod \"iptables-alerter-dtpsd\" (UID: \"33101463-3e59-49a7-91d4-947558c4b6c7\") " pod="openshift-network-operator/iptables-alerter-dtpsd"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215591    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96vjm\" (UniqueName: \"kubernetes.io/projected/5eda199b-9a46-4bdd-919e-dbacd4631428-kube-api-access-96vjm\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215614    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-multus-cni-dir\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8v2l\" (UniqueName: \"kubernetes.io/projected/76e6dbde-7988-46ac-91e5-872bcc2ee08c-kube-api-access-b8v2l\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-run-k8s-cni-cncf-io\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215680    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-multus-conf-dir\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215694    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5eda199b-9a46-4bdd-919e-dbacd4631428-cnibin\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215709    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5eda199b-9a46-4bdd-919e-dbacd4631428-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215747    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-system-cni-dir\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215762    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-sys\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215775    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-etc-openvswitch\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215788    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-log-socket\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.215806 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-env-overrides\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215832    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-sys-fs\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-var-lib-openvswitch\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-kubelet-config\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215886    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx8hz\" (UniqueName: \"kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz\") pod \"network-check-target-psbp5\" (UID: \"67e048f7-9b14-45bd-860a-8cc46c8ecd19\") " pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215906    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215927    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-socket-dir\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs9bk\" (UniqueName: \"kubernetes.io/projected/fe6a8da4-b859-441c-9001-7b14d9c8f065-kube-api-access-qs9bk\") pod \"node-resolver-qmjbp\" (UID: \"fe6a8da4-b859-441c-9001-7b14d9c8f065\") " pod="openshift-dns/node-resolver-qmjbp"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215960    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqdkr\" (UniqueName: \"kubernetes.io/projected/a8f329d8-6209-4a6c-8004-60a1514c92ec-kube-api-access-rqdkr\") pod \"node-ca-n89pr\" (UID: \"a8f329d8-6209-4a6c-8004-60a1514c92ec\") " pod="openshift-image-registry/node-ca-n89pr"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.215980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-device-dir\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216003    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/26bb3600-39ee-4ac3-9baa-b6fc520712a2-cni-binary-copy\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216031    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-run-multus-certs\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-sysconfig\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-cni-bin\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216081    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5eda199b-9a46-4bdd-919e-dbacd4631428-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216096    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-multus-socket-dir-parent\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.216452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216110    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-systemd\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-run-ovn-kubernetes\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216173    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4klc\" (UniqueName: \"kubernetes.io/projected/24465b9d-ca62-4506-933b-f0b4096ca5da-kube-api-access-n4klc\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d0820a3-2459-4599-91e6-6e45d8ebb6f3-agent-certs\") pod \"konnectivity-agent-9k9h7\" (UID: \"3d0820a3-2459-4599-91e6-6e45d8ebb6f3\") " pod="kube-system/konnectivity-agent-9k9h7"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216222    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-var-lib-kubelet\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/26bb3600-39ee-4ac3-9baa-b6fc520712a2-multus-daemon-config\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216266    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-host\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216289    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-kubelet\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216311    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-ovnkube-config\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5eda199b-9a46-4bdd-919e-dbacd4631428-system-cni-dir\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216360    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-run-netns\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216381    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-run-netns\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216425    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-ovnkube-script-lib\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216447    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-dbus\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33101463-3e59-49a7-91d4-947558c4b6c7-host-slash\") pod \"iptables-alerter-dtpsd\" (UID: \"33101463-3e59-49a7-91d4-947558c4b6c7\") " pod="openshift-network-operator/iptables-alerter-dtpsd"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-hostroot\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.216942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216518    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-sysctl-conf\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-systemd-units\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216549    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2rm8\" (UniqueName: \"kubernetes.io/projected/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-kube-api-access-j2rm8\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4mcf\" (UniqueName: \"kubernetes.io/projected/33101463-3e59-49a7-91d4-947558c4b6c7-kube-api-access-w4mcf\") pod \"iptables-alerter-dtpsd\" (UID: \"33101463-3e59-49a7-91d4-947558c4b6c7\") " pod="openshift-network-operator/iptables-alerter-dtpsd"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216585    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5eda199b-9a46-4bdd-919e-dbacd4631428-os-release\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-952gc\" (UniqueName: \"kubernetes.io/projected/26bb3600-39ee-4ac3-9baa-b6fc520712a2-kube-api-access-952gc\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216636    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-ovn-node-metrics-cert\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216683    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-registration-dir\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216697    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs\") pod \"network-metrics-daemon-zgg7r\" (UID: \"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19\") " pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-modprobe-d\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216725    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-kubernetes\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216747    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-var-lib-kubelet\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216763    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-tuned\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d0820a3-2459-4599-91e6-6e45d8ebb6f3-konnectivity-ca\") pod \"konnectivity-agent-9k9h7\" (UID: \"3d0820a3-2459-4599-91e6-6e45d8ebb6f3\") " pod="kube-system/konnectivity-agent-9k9h7"
Jul 24 18:44:56.217406 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.216790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5eda199b-9a46-4bdd-919e-dbacd4631428-cni-binary-copy\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.317879 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.317852    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/26bb3600-39ee-4ac3-9baa-b6fc520712a2-cni-binary-copy\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.318052 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.317897    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-run-multus-certs\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.318052 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.317914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-sysconfig\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.318052 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.317930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-cni-bin\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.318052 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.317971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-cni-bin\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.318052 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.317978    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-run-multus-certs\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.318052 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.317997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5eda199b-9a46-4bdd-919e-dbacd4631428-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.318052 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318011    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-sysconfig\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.318052 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318025    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-multus-socket-dir-parent\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-systemd\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318084    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-multus-socket-dir-parent\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-run-ovn-kubernetes\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318122    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-run-ovn-kubernetes\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4klc\" (UniqueName: \"kubernetes.io/projected/24465b9d-ca62-4506-933b-f0b4096ca5da-kube-api-access-n4klc\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318125    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5eda199b-9a46-4bdd-919e-dbacd4631428-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d0820a3-2459-4599-91e6-6e45d8ebb6f3-agent-certs\") pod \"konnectivity-agent-9k9h7\" (UID: \"3d0820a3-2459-4599-91e6-6e45d8ebb6f3\") " pod="kube-system/konnectivity-agent-9k9h7"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-var-lib-kubelet\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318183    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-systemd\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318221    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-var-lib-kubelet\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318243    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/26bb3600-39ee-4ac3-9baa-b6fc520712a2-multus-daemon-config\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318259    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-host\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318278    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-kubelet\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-ovnkube-config\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318329    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5eda199b-9a46-4bdd-919e-dbacd4631428-system-cni-dir\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-host\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.318465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318351    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-run-netns\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318373    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-run-netns\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-ovnkube-script-lib\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318437    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-dbus\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318461    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33101463-3e59-49a7-91d4-947558c4b6c7-host-slash\") pod \"iptables-alerter-dtpsd\" (UID: \"33101463-3e59-49a7-91d4-947558c4b6c7\") " pod="openshift-network-operator/iptables-alerter-dtpsd"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318479    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/26bb3600-39ee-4ac3-9baa-b6fc520712a2-cni-binary-copy\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318481    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-run-netns\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-hostroot\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318485    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-hostroot\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-sysctl-conf\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318496    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5eda199b-9a46-4bdd-919e-dbacd4631428-system-cni-dir\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318564    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33101463-3e59-49a7-91d4-947558c4b6c7-host-slash\") pod \"iptables-alerter-dtpsd\" (UID: \"33101463-3e59-49a7-91d4-947558c4b6c7\") " pod="openshift-network-operator/iptables-alerter-dtpsd"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-systemd-units\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-kubelet\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2rm8\" (UniqueName: \"kubernetes.io/projected/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-kube-api-access-j2rm8\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318621    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-systemd-units\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318622    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4mcf\" (UniqueName: \"kubernetes.io/projected/33101463-3e59-49a7-91d4-947558c4b6c7-kube-api-access-w4mcf\") pod \"iptables-alerter-dtpsd\" (UID: \"33101463-3e59-49a7-91d4-947558c4b6c7\") " pod="openshift-network-operator/iptables-alerter-dtpsd"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318694    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-dbus\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:44:56.319248 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318704    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-sysctl-conf\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-run-netns\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318661    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5eda199b-9a46-4bdd-919e-dbacd4631428-os-release\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318749    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5eda199b-9a46-4bdd-919e-dbacd4631428-os-release\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-952gc\" (UniqueName: \"kubernetes.io/projected/26bb3600-39ee-4ac3-9baa-b6fc520712a2-kube-api-access-952gc\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318781    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/26bb3600-39ee-4ac3-9baa-b6fc520712a2-multus-daemon-config\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-ovn-node-metrics-cert\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318902    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-registration-dir\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318971    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs\") pod \"network-metrics-daemon-zgg7r\" (UID: \"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19\") " pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.318972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-ovnkube-config\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319019    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-modprobe-d\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-kubernetes\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.319059    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319058    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-var-lib-kubelet\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.319061    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319071    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-ovnkube-script-lib\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.320072 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319113    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-kubernetes\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.319129    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs podName:7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19 nodeName:}" failed. No retries permitted until 2026-07-24 18:44:56.819110514 +0000 UTC m=+35.210061984 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs") pod "network-metrics-daemon-zgg7r" (UID: "7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319117    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-var-lib-kubelet\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319141    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-modprobe-d\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.319154    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret podName:c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3 nodeName:}" failed. No retries permitted until 2026-07-24 18:44:56.819138273 +0000 UTC m=+35.210089728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret") pod "global-pull-secret-syncer-jwsd9" (UID: "c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3") : object "kube-system"/"original-pull-secret" not registered
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319151    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-registration-dir\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319185    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-tuned\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d0820a3-2459-4599-91e6-6e45d8ebb6f3-konnectivity-ca\") pod \"konnectivity-agent-9k9h7\" (UID: \"3d0820a3-2459-4599-91e6-6e45d8ebb6f3\") " pod="kube-system/konnectivity-agent-9k9h7"
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319209    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319225    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5eda199b-9a46-4bdd-919e-dbacd4631428-cni-binary-copy\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319245    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5eda199b-9a46-4bdd-919e-dbacd4631428-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-var-lib-cni-bin\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-sysctl-d\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-lib-modules\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8f329d8-6209-4a6c-8004-60a1514c92ec-serviceca\") pod \"node-ca-n89pr\" (UID: \"a8f329d8-6209-4a6c-8004-60a1514c92ec\") " pod="openshift-image-registry/node-ca-n89pr"
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319374    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-node-log\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.320880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-cnibin\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-sysctl-d\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-slash\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319505    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-var-lib-cni-bin\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-run-systemd\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-cni-netd\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319548    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-node-log\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-etc-selinux\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfz9h\" (UniqueName: \"kubernetes.io/projected/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-kube-api-access-bfz9h\") pod \"network-metrics-daemon-zgg7r\" (UID: \"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19\") " pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319638    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe6a8da4-b859-441c-9001-7b14d9c8f065-tmp-dir\") pod \"node-resolver-qmjbp\" (UID: \"fe6a8da4-b859-441c-9001-7b14d9c8f065\") " pod="openshift-dns/node-resolver-qmjbp"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319661    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-var-lib-cni-multus\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319680    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-run\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319695    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-run-ovn\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319705    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-lib-modules\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319713    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fe6a8da4-b859-441c-9001-7b14d9c8f065-hosts-file\") pod \"node-resolver-qmjbp\" (UID: \"fe6a8da4-b859-441c-9001-7b14d9c8f065\") " pod="openshift-dns/node-resolver-qmjbp"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-cnibin\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-os-release\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.321829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319795    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-etc-kubernetes\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319802    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-os-release\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319823    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76e6dbde-7988-46ac-91e5-872bcc2ee08c-tmp\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319837    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fe6a8da4-b859-441c-9001-7b14d9c8f065-hosts-file\") pod \"node-resolver-qmjbp\" (UID: \"fe6a8da4-b859-441c-9001-7b14d9c8f065\") " pod="openshift-dns/node-resolver-qmjbp"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319752    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319848    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8f329d8-6209-4a6c-8004-60a1514c92ec-host\") pod \"node-ca-n89pr\" (UID: \"a8f329d8-6209-4a6c-8004-60a1514c92ec\") " pod="openshift-image-registry/node-ca-n89pr"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319872    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-slash\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319888    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8f329d8-6209-4a6c-8004-60a1514c92ec-host\") pod \"node-ca-n89pr\" (UID: \"a8f329d8-6209-4a6c-8004-60a1514c92ec\") " pod="openshift-image-registry/node-ca-n89pr"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319905    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-etc-kubernetes\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319970    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-etc-selinux\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.320005    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-run-systemd\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.320046    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-run\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.320081    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-var-lib-cni-multus\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.320114    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-run-ovn\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.319693    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-host-cni-netd\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.320329    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe6a8da4-b859-441c-9001-7b14d9c8f065-tmp-dir\") pod \"node-resolver-qmjbp\" (UID: \"fe6a8da4-b859-441c-9001-7b14d9c8f065\") " pod="openshift-dns/node-resolver-qmjbp"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.320424    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-run-openvswitch\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.320482    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/33101463-3e59-49a7-91d4-947558c4b6c7-iptables-alerter-script\") pod \"iptables-alerter-dtpsd\" (UID: \"33101463-3e59-49a7-91d4-947558c4b6c7\") " pod="openshift-network-operator/iptables-alerter-dtpsd"
Jul 24 18:44:56.322627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.320512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96vjm\" (UniqueName: \"kubernetes.io/projected/5eda199b-9a46-4bdd-919e-dbacd4631428-kube-api-access-96vjm\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.320617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d0820a3-2459-4599-91e6-6e45d8ebb6f3-konnectivity-ca\") pod \"konnectivity-agent-9k9h7\" (UID: \"3d0820a3-2459-4599-91e6-6e45d8ebb6f3\") " pod="kube-system/konnectivity-agent-9k9h7"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.320640    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-run-openvswitch\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.320742    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8f329d8-6209-4a6c-8004-60a1514c92ec-serviceca\") pod \"node-ca-n89pr\" (UID: \"a8f329d8-6209-4a6c-8004-60a1514c92ec\") " pod="openshift-image-registry/node-ca-n89pr"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.320866    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5eda199b-9a46-4bdd-919e-dbacd4631428-cni-binary-copy\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.320943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-multus-cni-dir\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321039    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-multus-cni-dir\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321129    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/33101463-3e59-49a7-91d4-947558c4b6c7-iptables-alerter-script\") pod \"iptables-alerter-dtpsd\" (UID: \"33101463-3e59-49a7-91d4-947558c4b6c7\") " pod="openshift-network-operator/iptables-alerter-dtpsd"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8v2l\" (UniqueName: \"kubernetes.io/projected/76e6dbde-7988-46ac-91e5-872bcc2ee08c-kube-api-access-b8v2l\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-run-k8s-cni-cncf-io\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321466    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5eda199b-9a46-4bdd-919e-dbacd4631428-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321506    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-host-run-k8s-cni-cncf-io\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-multus-conf-dir\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321572    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5eda199b-9a46-4bdd-919e-dbacd4631428-cnibin\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-multus-conf-dir\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321622    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5eda199b-9a46-4bdd-919e-dbacd4631428-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-system-cni-dir\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.323481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321684    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5eda199b-9a46-4bdd-919e-dbacd4631428-cnibin\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-sys\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321729    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-etc-openvswitch\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321741    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26bb3600-39ee-4ac3-9baa-b6fc520712a2-system-cni-dir\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-log-socket\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321771    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76e6dbde-7988-46ac-91e5-872bcc2ee08c-sys\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321780    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-env-overrides\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321814    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-log-socket\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321783    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-etc-openvswitch\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321808    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-sys-fs\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-var-lib-openvswitch\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321873    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-sys-fs\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-kubelet-config\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321899    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-var-lib-openvswitch\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321905    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx8hz\" (UniqueName: \"kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz\") pod \"network-check-target-psbp5\" (UID: \"67e048f7-9b14-45bd-860a-8cc46c8ecd19\") " pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-socket-dir\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.324011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.321989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-kubelet-config\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:44:56.324562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.322001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs9bk\" (UniqueName: \"kubernetes.io/projected/fe6a8da4-b859-441c-9001-7b14d9c8f065-kube-api-access-qs9bk\") pod \"node-resolver-qmjbp\" (UID: \"fe6a8da4-b859-441c-9001-7b14d9c8f065\") " pod="openshift-dns/node-resolver-qmjbp"
Jul 24 18:44:56.324562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.322027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqdkr\" (UniqueName: \"kubernetes.io/projected/a8f329d8-6209-4a6c-8004-60a1514c92ec-kube-api-access-rqdkr\") pod \"node-ca-n89pr\" (UID: \"a8f329d8-6209-4a6c-8004-60a1514c92ec\") " pod="openshift-image-registry/node-ca-n89pr"
Jul 24 18:44:56.324562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.322038    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.324562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.322051    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-device-dir\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.324562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.322144    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-env-overrides\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.324562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.322173    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-socket-dir\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.324562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.322293    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/24465b9d-ca62-4506-933b-f0b4096ca5da-device-dir\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.324562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.322570    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5eda199b-9a46-4bdd-919e-dbacd4631428-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.324562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.323671    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76e6dbde-7988-46ac-91e5-872bcc2ee08c-etc-tuned\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.324562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.323766    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76e6dbde-7988-46ac-91e5-872bcc2ee08c-tmp\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.324562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.323975    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d0820a3-2459-4599-91e6-6e45d8ebb6f3-agent-certs\") pod \"konnectivity-agent-9k9h7\" (UID: \"3d0820a3-2459-4599-91e6-6e45d8ebb6f3\") " pod="kube-system/konnectivity-agent-9k9h7"
Jul 24 18:44:56.324562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.324055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-ovn-node-metrics-cert\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.328382 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.328208    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 18:44:56.328382 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.328233    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 18:44:56.328382 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.328246    2563 projected.go:196] Error preparing data for projected volume kube-api-access-vx8hz for pod openshift-network-diagnostics/network-check-target-psbp5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 18:44:56.328382 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.328314    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz podName:67e048f7-9b14-45bd-860a-8cc46c8ecd19 nodeName:}" failed. No retries permitted until 2026-07-24 18:44:56.828297549 +0000 UTC m=+35.219249004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vx8hz" (UniqueName: "kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz") pod "network-check-target-psbp5" (UID: "67e048f7-9b14-45bd-860a-8cc46c8ecd19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 18:44:56.328668 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.328480    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4klc\" (UniqueName: \"kubernetes.io/projected/24465b9d-ca62-4506-933b-f0b4096ca5da-kube-api-access-n4klc\") pod \"aws-ebs-csi-driver-node-gglrb\" (UID: \"24465b9d-ca62-4506-933b-f0b4096ca5da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.329363 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.329329    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2rm8\" (UniqueName: \"kubernetes.io/projected/e5a77bc5-eeda-4e10-872e-190ad4ffeff5-kube-api-access-j2rm8\") pod \"ovnkube-node-889p5\" (UID: \"e5a77bc5-eeda-4e10-872e-190ad4ffeff5\") " pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.329363 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.329354    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-952gc\" (UniqueName: \"kubernetes.io/projected/26bb3600-39ee-4ac3-9baa-b6fc520712a2-kube-api-access-952gc\") pod \"multus-fmsn5\" (UID: \"26bb3600-39ee-4ac3-9baa-b6fc520712a2\") " pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.329563 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.329540    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4mcf\" (UniqueName: \"kubernetes.io/projected/33101463-3e59-49a7-91d4-947558c4b6c7-kube-api-access-w4mcf\") pod \"iptables-alerter-dtpsd\" (UID: \"33101463-3e59-49a7-91d4-947558c4b6c7\") " pod="openshift-network-operator/iptables-alerter-dtpsd"
Jul 24 18:44:56.329821 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.329800    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96vjm\" (UniqueName: \"kubernetes.io/projected/5eda199b-9a46-4bdd-919e-dbacd4631428-kube-api-access-96vjm\") pod \"multus-additional-cni-plugins-tpk4s\" (UID: \"5eda199b-9a46-4bdd-919e-dbacd4631428\") " pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.330126 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.330103    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfz9h\" (UniqueName: \"kubernetes.io/projected/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-kube-api-access-bfz9h\") pod \"network-metrics-daemon-zgg7r\" (UID: \"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19\") " pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:44:56.330809 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.330769    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8v2l\" (UniqueName: \"kubernetes.io/projected/76e6dbde-7988-46ac-91e5-872bcc2ee08c-kube-api-access-b8v2l\") pod \"tuned-qm2vs\" (UID: \"76e6dbde-7988-46ac-91e5-872bcc2ee08c\") " pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.331074 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.331057    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqdkr\" (UniqueName: \"kubernetes.io/projected/a8f329d8-6209-4a6c-8004-60a1514c92ec-kube-api-access-rqdkr\") pod \"node-ca-n89pr\" (UID: \"a8f329d8-6209-4a6c-8004-60a1514c92ec\") " pod="openshift-image-registry/node-ca-n89pr"
Jul 24 18:44:56.331173 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.331157    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs9bk\" (UniqueName: \"kubernetes.io/projected/fe6a8da4-b859-441c-9001-7b14d9c8f065-kube-api-access-qs9bk\") pod \"node-resolver-qmjbp\" (UID: \"fe6a8da4-b859-441c-9001-7b14d9c8f065\") " pod="openshift-dns/node-resolver-qmjbp"
Jul 24 18:44:56.336129 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.336115    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qmjbp"
Jul 24 18:44:56.342792 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.342773    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9k9h7"
Jul 24 18:44:56.347009 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:56.346989    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe6a8da4_b859_441c_9001_7b14d9c8f065.slice/crio-bce03c720aa89e7bb417d0e56b78df3680f1e54eff9f4430bcfd2275e7969767 WatchSource:0}: Error finding container bce03c720aa89e7bb417d0e56b78df3680f1e54eff9f4430bcfd2275e7969767: Status 404 returned error can't find the container with id bce03c720aa89e7bb417d0e56b78df3680f1e54eff9f4430bcfd2275e7969767
Jul 24 18:44:56.349227 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.349211    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tpk4s"
Jul 24 18:44:56.350734 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:56.350714    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d0820a3_2459_4599_91e6_6e45d8ebb6f3.slice/crio-8372d17b11437d9bf1dd865c72708e4044d9aa9fb2e2b65e238974c64df001a3 WatchSource:0}: Error finding container 8372d17b11437d9bf1dd865c72708e4044d9aa9fb2e2b65e238974c64df001a3: Status 404 returned error can't find the container with id 8372d17b11437d9bf1dd865c72708e4044d9aa9fb2e2b65e238974c64df001a3
Jul 24 18:44:56.354141 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.354123    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fmsn5"
Jul 24 18:44:56.359771 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.359751    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb"
Jul 24 18:44:56.361458 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:56.361427    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eda199b_9a46_4bdd_919e_dbacd4631428.slice/crio-2ca0d04dedff89b8b5a59c43cdbf8d147754b93b229c9d61121fd5da9572b916 WatchSource:0}: Error finding container 2ca0d04dedff89b8b5a59c43cdbf8d147754b93b229c9d61121fd5da9572b916: Status 404 returned error can't find the container with id 2ca0d04dedff89b8b5a59c43cdbf8d147754b93b229c9d61121fd5da9572b916
Jul 24 18:44:56.362464 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:56.362439    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26bb3600_39ee_4ac3_9baa_b6fc520712a2.slice/crio-9f75710471a0138de30b0aa0b4f0c3a35d28f3f99c69f5fd8a257654b5f2b6f8 WatchSource:0}: Error finding container 9f75710471a0138de30b0aa0b4f0c3a35d28f3f99c69f5fd8a257654b5f2b6f8: Status 404 returned error can't find the container with id 9f75710471a0138de30b0aa0b4f0c3a35d28f3f99c69f5fd8a257654b5f2b6f8
Jul 24 18:44:56.364219 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.364199    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qm2vs"
Jul 24 18:44:56.369627 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.369599    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n89pr"
Jul 24 18:44:56.373908 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:56.373888    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76e6dbde_7988_46ac_91e5_872bcc2ee08c.slice/crio-af52e821237f6c63d33471d2c382549fcde5248307463e4ad9f00927b5aa34a6 WatchSource:0}: Error finding container af52e821237f6c63d33471d2c382549fcde5248307463e4ad9f00927b5aa34a6: Status 404 returned error can't find the container with id af52e821237f6c63d33471d2c382549fcde5248307463e4ad9f00927b5aa34a6
Jul 24 18:44:56.374788 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.374770    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dtpsd"
Jul 24 18:44:56.379561 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.379543    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:44:56.381651 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:56.381625    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8f329d8_6209_4a6c_8004_60a1514c92ec.slice/crio-d6c62ac52c6f51ddc6e4600964385171809fd1a2377998f9454c439574f68494 WatchSource:0}: Error finding container d6c62ac52c6f51ddc6e4600964385171809fd1a2377998f9454c439574f68494: Status 404 returned error can't find the container with id d6c62ac52c6f51ddc6e4600964385171809fd1a2377998f9454c439574f68494
Jul 24 18:44:56.387045 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:56.387024    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33101463_3e59_49a7_91d4_947558c4b6c7.slice/crio-56e67984d45a5b75804236681bd682e904c2749c65c6eb0e3e722bb2e506d24a WatchSource:0}: Error finding container 56e67984d45a5b75804236681bd682e904c2749c65c6eb0e3e722bb2e506d24a: Status 404 returned error can't find the container with id 56e67984d45a5b75804236681bd682e904c2749c65c6eb0e3e722bb2e506d24a
Jul 24 18:44:56.391806 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:44:56.391787    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5a77bc5_eeda_4e10_872e_190ad4ffeff5.slice/crio-744d5fb3c0241530dad99a09b68c0b3ae788e7e49384cfb51d8eb3f18f90be18 WatchSource:0}: Error finding container 744d5fb3c0241530dad99a09b68c0b3ae788e7e49384cfb51d8eb3f18f90be18: Status 404 returned error can't find the container with id 744d5fb3c0241530dad99a09b68c0b3ae788e7e49384cfb51d8eb3f18f90be18
Jul 24 18:44:56.825741 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.825703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:44:56.825741 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.825753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs\") pod \"network-metrics-daemon-zgg7r\" (UID: \"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19\") " pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:44:56.825982 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.825869    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 18:44:56.825982 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.825925    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 18:44:56.825982 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.825948    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret podName:c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3 nodeName:}" failed. No retries permitted until 2026-07-24 18:44:57.825914936 +0000 UTC m=+36.216866394 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret") pod "global-pull-secret-syncer-jwsd9" (UID: "c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3") : object "kube-system"/"original-pull-secret" not registered
Jul 24 18:44:56.825982 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.825976    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs podName:7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19 nodeName:}" failed. No retries permitted until 2026-07-24 18:44:57.825960946 +0000 UTC m=+36.216912418 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs") pod "network-metrics-daemon-zgg7r" (UID: "7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 18:44:56.926125 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:56.926082    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx8hz\" (UniqueName: \"kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz\") pod \"network-check-target-psbp5\" (UID: \"67e048f7-9b14-45bd-860a-8cc46c8ecd19\") " pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:44:56.926374 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.926325    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 18:44:56.926374 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.926357    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 18:44:56.926374 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.926372    2563 projected.go:196] Error preparing data for projected volume kube-api-access-vx8hz for pod openshift-network-diagnostics/network-check-target-psbp5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 18:44:56.926601 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:56.926455    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz podName:67e048f7-9b14-45bd-860a-8cc46c8ecd19 nodeName:}" failed. No retries permitted until 2026-07-24 18:44:57.926436669 +0000 UTC m=+36.317388142 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx8hz" (UniqueName: "kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz") pod "network-check-target-psbp5" (UID: "67e048f7-9b14-45bd-860a-8cc46c8ecd19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 18:44:57.086671 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.086545    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 18:39:56 +0000 UTC" deadline="2028-01-07 07:42:44.175066014 +0000 UTC"
Jul 24 18:44:57.086671 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.086580    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12756h57m47.088490528s"
Jul 24 18:44:57.199590 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.198940    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:44:57.199590 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:57.199081    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:44:57.199590 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.199159    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:44:57.199590 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:57.199227    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:44:57.291240 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.291201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qmjbp" event={"ID":"fe6a8da4-b859-441c-9001-7b14d9c8f065","Type":"ContainerStarted","Data":"bce03c720aa89e7bb417d0e56b78df3680f1e54eff9f4430bcfd2275e7969767"}
Jul 24 18:44:57.295287 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.295192    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dtpsd" event={"ID":"33101463-3e59-49a7-91d4-947558c4b6c7","Type":"ContainerStarted","Data":"56e67984d45a5b75804236681bd682e904c2749c65c6eb0e3e722bb2e506d24a"}
Jul 24 18:44:57.307883 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.307849    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n89pr" event={"ID":"a8f329d8-6209-4a6c-8004-60a1514c92ec","Type":"ContainerStarted","Data":"d6c62ac52c6f51ddc6e4600964385171809fd1a2377998f9454c439574f68494"}
Jul 24 18:44:57.313426 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.313380    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qm2vs" event={"ID":"76e6dbde-7988-46ac-91e5-872bcc2ee08c","Type":"ContainerStarted","Data":"af52e821237f6c63d33471d2c382549fcde5248307463e4ad9f00927b5aa34a6"}
Jul 24 18:44:57.318508 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.318475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb" event={"ID":"24465b9d-ca62-4506-933b-f0b4096ca5da","Type":"ContainerStarted","Data":"8649316b8634b98ee3e3b636d570ee9061fdfe9ffb61f73d16dacdcbc2477ecf"}
Jul 24 18:44:57.330092 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.330054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fmsn5" event={"ID":"26bb3600-39ee-4ac3-9baa-b6fc520712a2","Type":"ContainerStarted","Data":"9f75710471a0138de30b0aa0b4f0c3a35d28f3f99c69f5fd8a257654b5f2b6f8"}
Jul 24 18:44:57.335481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.335447    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9k9h7" event={"ID":"3d0820a3-2459-4599-91e6-6e45d8ebb6f3","Type":"ContainerStarted","Data":"8372d17b11437d9bf1dd865c72708e4044d9aa9fb2e2b65e238974c64df001a3"}
Jul 24 18:44:57.354926 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.354767    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-889p5" event={"ID":"e5a77bc5-eeda-4e10-872e-190ad4ffeff5","Type":"ContainerStarted","Data":"744d5fb3c0241530dad99a09b68c0b3ae788e7e49384cfb51d8eb3f18f90be18"}
Jul 24 18:44:57.357718 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.357634    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpk4s" event={"ID":"5eda199b-9a46-4bdd-919e-dbacd4631428","Type":"ContainerStarted","Data":"2ca0d04dedff89b8b5a59c43cdbf8d147754b93b229c9d61121fd5da9572b916"}
Jul 24 18:44:57.835945 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.835905    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:44:57.836154 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.835961    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs\") pod \"network-metrics-daemon-zgg7r\" (UID: \"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19\") " pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:44:57.836154 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:57.836112    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 18:44:57.836383 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:57.836178    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs podName:7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19 nodeName:}" failed. No retries permitted until 2026-07-24 18:44:59.836157495 +0000 UTC m=+38.227108950 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs") pod "network-metrics-daemon-zgg7r" (UID: "7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 18:44:57.836593 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:57.836543    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 18:44:57.836684 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:57.836602    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret podName:c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3 nodeName:}" failed. No retries permitted until 2026-07-24 18:44:59.836584233 +0000 UTC m=+38.227535694 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret") pod "global-pull-secret-syncer-jwsd9" (UID: "c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3") : object "kube-system"/"original-pull-secret" not registered
Jul 24 18:44:57.937308 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:57.937270    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx8hz\" (UniqueName: \"kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz\") pod \"network-check-target-psbp5\" (UID: \"67e048f7-9b14-45bd-860a-8cc46c8ecd19\") " pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:44:57.937499 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:57.937474    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 18:44:57.937499 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:57.937496    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 18:44:57.937600 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:57.937510    2563 projected.go:196] Error preparing data for projected volume kube-api-access-vx8hz for pod openshift-network-diagnostics/network-check-target-psbp5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 18:44:57.937600 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:57.937572    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz podName:67e048f7-9b14-45bd-860a-8cc46c8ecd19 nodeName:}" failed. No retries permitted until 2026-07-24 18:44:59.937552434 +0000 UTC m=+38.328503911 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx8hz" (UniqueName: "kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz") pod "network-check-target-psbp5" (UID: "67e048f7-9b14-45bd-860a-8cc46c8ecd19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 18:44:58.087042 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:58.086926    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 18:39:56 +0000 UTC" deadline="2028-04-02 18:14:57.94433818 +0000 UTC"
Jul 24 18:44:58.087042 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:58.086991    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14831h29m59.857352016s"
Jul 24 18:44:58.200934 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:58.200901    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:44:58.201108 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:58.201048    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:44:59.198301 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:59.198273    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:44:59.198798 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:59.198472    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:44:59.198798 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:59.198559    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:44:59.198984 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:59.198957    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:44:59.853749 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:59.853712    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:44:59.853945 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:59.853762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs\") pod \"network-metrics-daemon-zgg7r\" (UID: \"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19\") " pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:44:59.853945 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:59.853890    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 18:44:59.853945 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:59.853943    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs podName:7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19 nodeName:}" failed. No retries permitted until 2026-07-24 18:45:03.853928739 +0000 UTC m=+42.244880193 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs") pod "network-metrics-daemon-zgg7r" (UID: "7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 18:44:59.854340 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:59.854322    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 18:44:59.854437 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:59.854380    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret podName:c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3 nodeName:}" failed. No retries permitted until 2026-07-24 18:45:03.854362998 +0000 UTC m=+42.245314475 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret") pod "global-pull-secret-syncer-jwsd9" (UID: "c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3") : object "kube-system"/"original-pull-secret" not registered
Jul 24 18:44:59.954270 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:44:59.954226    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx8hz\" (UniqueName: \"kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz\") pod \"network-check-target-psbp5\" (UID: \"67e048f7-9b14-45bd-860a-8cc46c8ecd19\") " pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:44:59.954499 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:59.954408    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 18:44:59.954499 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:59.954430    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 18:44:59.954499 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:59.954442    2563 projected.go:196] Error preparing data for projected volume kube-api-access-vx8hz for pod openshift-network-diagnostics/network-check-target-psbp5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 18:44:59.954668 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:44:59.954506    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz podName:67e048f7-9b14-45bd-860a-8cc46c8ecd19 nodeName:}" failed. No retries permitted until 2026-07-24 18:45:03.954490429 +0000 UTC m=+42.345441887 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx8hz" (UniqueName: "kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz") pod "network-check-target-psbp5" (UID: "67e048f7-9b14-45bd-860a-8cc46c8ecd19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 18:45:00.200380 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:00.198267    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:00.200380 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:00.198411    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:00.778646 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:00.778323    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 18:45:01.198764 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:01.198098    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:01.198764 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:01.198246    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:01.198764 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:01.198621    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:01.198764 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:01.198719    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:01.411507 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:01.411298    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 18:45:02.199950 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:02.199440    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:02.199950 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:02.199560    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:03.198282 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:03.198252    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:03.198282 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:03.198293    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:03.198761 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:03.198374    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:03.198761 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:03.198536    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:03.885764 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:03.885722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:03.885764 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:03.885773    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs\") pod \"network-metrics-daemon-zgg7r\" (UID: \"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19\") " pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:03.886004 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:03.885898    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 18:45:03.886004 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:03.885939    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 18:45:03.886004 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:03.885967    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret podName:c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3 nodeName:}" failed. No retries permitted until 2026-07-24 18:45:11.885952594 +0000 UTC m=+50.276904053 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret") pod "global-pull-secret-syncer-jwsd9" (UID: "c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3") : object "kube-system"/"original-pull-secret" not registered
Jul 24 18:45:03.886004 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:03.885993    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs podName:7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19 nodeName:}" failed. No retries permitted until 2026-07-24 18:45:11.885977046 +0000 UTC m=+50.276928524 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs") pod "network-metrics-daemon-zgg7r" (UID: "7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 18:45:03.986408 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:03.986353    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx8hz\" (UniqueName: \"kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz\") pod \"network-check-target-psbp5\" (UID: \"67e048f7-9b14-45bd-860a-8cc46c8ecd19\") " pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:03.986597 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:03.986577    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 18:45:03.986597 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:03.986596    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 18:45:03.986731 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:03.986605    2563 projected.go:196] Error preparing data for projected volume kube-api-access-vx8hz for pod openshift-network-diagnostics/network-check-target-psbp5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 18:45:03.986731 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:03.986652    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz podName:67e048f7-9b14-45bd-860a-8cc46c8ecd19 nodeName:}" failed. No retries permitted until 2026-07-24 18:45:11.986638785 +0000 UTC m=+50.377590243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx8hz" (UniqueName: "kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz") pod "network-check-target-psbp5" (UID: "67e048f7-9b14-45bd-860a-8cc46c8ecd19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 18:45:04.198678 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:04.198625    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:04.199152 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:04.198751    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:05.198990 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:05.198339    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:05.198990 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:05.198351    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:05.198990 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:05.198496    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:05.198990 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:05.198948    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:06.198210 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.198168    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:06.198379 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:06.198293    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:06.706179 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.706144    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cxgj5"]
Jul 24 18:45:06.709354 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.709326    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.712976 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.712951    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 18:45:06.713103 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.713022    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 18:45:06.714197 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.713913    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 18:45:06.714197 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.713978    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l828c\""
Jul 24 18:45:06.714197 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.714004    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 18:45:06.714197 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.714072    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 18:45:06.714197 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.714134    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 18:45:06.810016 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.809971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/076e1bdd-08e1-4c23-8029-91474ae2132a-sys\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.810016 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.810018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/076e1bdd-08e1-4c23-8029-91474ae2132a-metrics-client-ca\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.810255 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.810073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-accelerators-collector-config\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.810255 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.810094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-textfile\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.810255 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.810120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/076e1bdd-08e1-4c23-8029-91474ae2132a-root\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.810255 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.810144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.810255 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.810167    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-tls\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.810255 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.810186    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-wtmp\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.810255 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.810208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgbwf\" (UniqueName: \"kubernetes.io/projected/076e1bdd-08e1-4c23-8029-91474ae2132a-kube-api-access-rgbwf\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.910991 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.910908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-accelerators-collector-config\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.910991 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.910951    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-textfile\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.910991 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.910976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/076e1bdd-08e1-4c23-8029-91474ae2132a-root\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.911240 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.911007    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.911240 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.911044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-tls\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.911240 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.911061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-wtmp\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.911240 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.911097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgbwf\" (UniqueName: \"kubernetes.io/projected/076e1bdd-08e1-4c23-8029-91474ae2132a-kube-api-access-rgbwf\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.911240 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.911128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/076e1bdd-08e1-4c23-8029-91474ae2132a-sys\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.911240 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.911151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/076e1bdd-08e1-4c23-8029-91474ae2132a-metrics-client-ca\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.911858 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.911832    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/076e1bdd-08e1-4c23-8029-91474ae2132a-metrics-client-ca\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.912209 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.912190    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-accelerators-collector-config\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.912435 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.912417    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-textfile\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.912540 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.912517    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/076e1bdd-08e1-4c23-8029-91474ae2132a-root\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.912594 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.912536    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-wtmp\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.912863 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.912843    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/076e1bdd-08e1-4c23-8029-91474ae2132a-sys\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.915935 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.915913    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-tls\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.915935 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.915924    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/076e1bdd-08e1-4c23-8029-91474ae2132a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:06.920705 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:06.920684    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgbwf\" (UniqueName: \"kubernetes.io/projected/076e1bdd-08e1-4c23-8029-91474ae2132a-kube-api-access-rgbwf\") pod \"node-exporter-cxgj5\" (UID: \"076e1bdd-08e1-4c23-8029-91474ae2132a\") " pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:07.021411 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:07.021369    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cxgj5"
Jul 24 18:45:07.199534 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:07.198994    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:07.199534 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:07.198999    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:07.199534 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:07.199123    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:07.199534 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:07.199210    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:08.198218 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:08.198183    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:08.198784 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:08.198322    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:09.198404 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:09.198341    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:09.198880 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:09.198487    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:09.198880 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:09.198526    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:09.198880 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:09.198609    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:10.198468 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:10.198436    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:10.198933 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:10.198563    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:11.198426 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:11.198379    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:11.198601 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:11.198527    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:11.198601 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:11.198577    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:11.199010 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:11.198694    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:11.950273 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:11.950233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:11.950506 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:11.950288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs\") pod \"network-metrics-daemon-zgg7r\" (UID: \"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19\") " pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:11.950506 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:11.950353    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 18:45:11.950506 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:11.950415    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 18:45:11.950506 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:11.950454    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret podName:c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3 nodeName:}" failed. No retries permitted until 2026-07-24 18:45:27.95043531 +0000 UTC m=+66.341386765 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret") pod "global-pull-secret-syncer-jwsd9" (UID: "c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3") : object "kube-system"/"original-pull-secret" not registered
Jul 24 18:45:11.950506 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:11.950471    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs podName:7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19 nodeName:}" failed. No retries permitted until 2026-07-24 18:45:27.950464563 +0000 UTC m=+66.341416018 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs") pod "network-metrics-daemon-zgg7r" (UID: "7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 18:45:12.050791 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:12.050737    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx8hz\" (UniqueName: \"kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz\") pod \"network-check-target-psbp5\" (UID: \"67e048f7-9b14-45bd-860a-8cc46c8ecd19\") " pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:12.050964 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:12.050913    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 18:45:12.050964 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:12.050933    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 18:45:12.050964 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:12.050947    2563 projected.go:196] Error preparing data for projected volume kube-api-access-vx8hz for pod openshift-network-diagnostics/network-check-target-psbp5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 18:45:12.051084 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:12.051017    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz podName:67e048f7-9b14-45bd-860a-8cc46c8ecd19 nodeName:}" failed. No retries permitted until 2026-07-24 18:45:28.050996598 +0000 UTC m=+66.441948059 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx8hz" (UniqueName: "kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz") pod "network-check-target-psbp5" (UID: "67e048f7-9b14-45bd-860a-8cc46c8ecd19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 18:45:12.200042 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:12.199745    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:12.200042 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:12.199861    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:13.198731 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:13.198686    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:13.198901 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:13.198736    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:13.198901 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:13.198833    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:13.199034 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:13.198967    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:14.199386 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:14.199352    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:14.200018 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:14.199512    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:14.567099 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:45:14.564863    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod076e1bdd_08e1_4c23_8029_91474ae2132a.slice/crio-d3ee4cfc634d048dac5de02a5bf0b846d3021176a9f324edce3b4292c7c4bd9b WatchSource:0}: Error finding container d3ee4cfc634d048dac5de02a5bf0b846d3021176a9f324edce3b4292c7c4bd9b: Status 404 returned error can't find the container with id d3ee4cfc634d048dac5de02a5bf0b846d3021176a9f324edce3b4292c7c4bd9b
Jul 24 18:45:15.198466 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.198200    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:15.198643 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:15.198615    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:15.198728 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.198228    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:15.198811 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:15.198790    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:15.400556 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.400296    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb" event={"ID":"24465b9d-ca62-4506-933b-f0b4096ca5da","Type":"ContainerStarted","Data":"c97abdcb10792cdfd3bc84dcb4373ebaabb149812eaf464e4e2eef40662887f3"}
Jul 24 18:45:15.402140 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.402116    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fmsn5" event={"ID":"26bb3600-39ee-4ac3-9baa-b6fc520712a2","Type":"ContainerStarted","Data":"3f51762a076cea63a2fe04b1852a95b4fc8562d9b9e39b6a0bf7b62e4576022c"}
Jul 24 18:45:15.403829 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.403799    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9k9h7" event={"ID":"3d0820a3-2459-4599-91e6-6e45d8ebb6f3","Type":"ContainerStarted","Data":"a109afcfba477bf26b1925e274c32fabec9537bf2bb836542d2f774edf446bb5"}
Jul 24 18:45:15.406696 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.406676    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 18:45:15.407074 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.407052    2563 generic.go:356] "Generic (PLEG): container finished" podID="e5a77bc5-eeda-4e10-872e-190ad4ffeff5" containerID="880591db7c87ccf3b487948bea669b3883bf2167635dd26933c876f943f25433" exitCode=1
Jul 24 18:45:15.407152 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.407121    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-889p5" event={"ID":"e5a77bc5-eeda-4e10-872e-190ad4ffeff5","Type":"ContainerStarted","Data":"ace1adaaa50254760a2d1ff9ad2bc1b09347eb88ab9ff39ab55ce867f758773d"}
Jul 24 18:45:15.407221 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.407193    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-889p5" event={"ID":"e5a77bc5-eeda-4e10-872e-190ad4ffeff5","Type":"ContainerStarted","Data":"70e7caf331245a4e81375e228c8c1152715184d4cec0d7bce04c9b932d5fcaf7"}
Jul 24 18:45:15.407283 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.407231    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-889p5" event={"ID":"e5a77bc5-eeda-4e10-872e-190ad4ffeff5","Type":"ContainerDied","Data":"880591db7c87ccf3b487948bea669b3883bf2167635dd26933c876f943f25433"}
Jul 24 18:45:15.407335 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.407284    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-889p5" event={"ID":"e5a77bc5-eeda-4e10-872e-190ad4ffeff5","Type":"ContainerStarted","Data":"072ceef58c8201ceb772a54a9056e6ab761f3d379caefc99b9d6ebc4dcdcba5a"}
Jul 24 18:45:15.408867 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.408843    2563 generic.go:356] "Generic (PLEG): container finished" podID="5eda199b-9a46-4bdd-919e-dbacd4631428" containerID="3836c01a8d6fb2813f158c962f675c5f8624dd2d2004a4e52f92914537854081" exitCode=0
Jul 24 18:45:15.409117 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.409093    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpk4s" event={"ID":"5eda199b-9a46-4bdd-919e-dbacd4631428","Type":"ContainerDied","Data":"3836c01a8d6fb2813f158c962f675c5f8624dd2d2004a4e52f92914537854081"}
Jul 24 18:45:15.411632 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.411604    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qmjbp" event={"ID":"fe6a8da4-b859-441c-9001-7b14d9c8f065","Type":"ContainerStarted","Data":"c9063ec2b56c1564ae77c1609419a8bff6ab9ab12f0ea4aaee8b3872cd508d6e"}
Jul 24 18:45:15.413723 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.413698    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cxgj5" event={"ID":"076e1bdd-08e1-4c23-8029-91474ae2132a","Type":"ContainerStarted","Data":"d3ee4cfc634d048dac5de02a5bf0b846d3021176a9f324edce3b4292c7c4bd9b"}
Jul 24 18:45:15.415902 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.415884    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal_49c209b3e6e7d8404ad2a977c0bb87c9/setup/1.log"
Jul 24 18:45:15.416010 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.415918    2563 generic.go:356] "Generic (PLEG): container finished" podID="49c209b3e6e7d8404ad2a977c0bb87c9" containerID="5074cc7535351c11616a270b708baf4c863e0d209bb38770eef48341f96b4a36" exitCode=0
Jul 24 18:45:15.416065 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.416000    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" event={"ID":"49c209b3e6e7d8404ad2a977c0bb87c9","Type":"ContainerDied","Data":"5074cc7535351c11616a270b708baf4c863e0d209bb38770eef48341f96b4a36"}
Jul 24 18:45:15.416065 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.416060    2563 scope.go:117] "RemoveContainer" containerID="5181529562ed83993a121ee557d5a60fb4454bd55b60c04c7a28ceaf2d3f8990"
Jul 24 18:45:15.416319 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.416301    2563 scope.go:117] "RemoveContainer" containerID="5181529562ed83993a121ee557d5a60fb4454bd55b60c04c7a28ceaf2d3f8990"
Jul 24 18:45:15.418056 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.418033    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n89pr" event={"ID":"a8f329d8-6209-4a6c-8004-60a1514c92ec","Type":"ContainerStarted","Data":"b0badfb2c6e8a1f9705b6611984ffb6302776e593268ab7465e09081e02bab11"}
Jul 24 18:45:15.419877 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.419859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qm2vs" event={"ID":"76e6dbde-7988-46ac-91e5-872bcc2ee08c","Type":"ContainerStarted","Data":"f03901bb5d286680859d135f0ce4916a13a9614d0211dfc0a1c43865dc25d51e"}
Jul 24 18:45:15.422774 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.422732    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fmsn5" podStartSLOduration=8.179767653 podStartE2EDuration="26.42271671s" totalImagesPullingTime="18.242949057s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:44:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:45:15.421687695 +0000 UTC m=+53.812639195" watchObservedRunningTime="2026-07-24 18:45:15.42271671 +0000 UTC m=+53.813668189"
Jul 24 18:45:15.437584 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.437539    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-n89pr" podStartSLOduration=8.256743596 podStartE2EDuration="26.437530915s" totalImagesPullingTime="18.180787319s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:44:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:45:15.437454371 +0000 UTC m=+53.828405848" watchObservedRunningTime="2026-07-24 18:45:15.437530915 +0000 UTC m=+53.828482392"
Jul 24 18:45:15.485289 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:15.485255    2563 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal_openshift-machine-config-operator_49c209b3e6e7d8404ad2a977c0bb87c9_1 in pod sandbox 4700a9d706a6b52dc763af2f6f7e79b91e528d480e522186ec9f93221ed8cdea from index: no such id: '5181529562ed83993a121ee557d5a60fb4454bd55b60c04c7a28ceaf2d3f8990'" containerID="5181529562ed83993a121ee557d5a60fb4454bd55b60c04c7a28ceaf2d3f8990"
Jul 24 18:45:15.485355 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.485299    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5181529562ed83993a121ee557d5a60fb4454bd55b60c04c7a28ceaf2d3f8990"} err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal_openshift-machine-config-operator_49c209b3e6e7d8404ad2a977c0bb87c9_1 in pod sandbox 4700a9d706a6b52dc763af2f6f7e79b91e528d480e522186ec9f93221ed8cdea from index: no such id: '5181529562ed83993a121ee557d5a60fb4454bd55b60c04c7a28ceaf2d3f8990'"
Jul 24 18:45:15.501453 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.501413    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qmjbp" podStartSLOduration=9.285802075 podStartE2EDuration="27.501383708s" totalImagesPullingTime="18.215581633s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:44:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:45:15.501235065 +0000 UTC m=+53.892186541" watchObservedRunningTime="2026-07-24 18:45:15.501383708 +0000 UTC m=+53.892335185"
Jul 24 18:45:15.522773 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.522726    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qm2vs" podStartSLOduration=8.332425449 podStartE2EDuration="26.522705375s" totalImagesPullingTime="18.190279926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:44:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:45:15.522022537 +0000 UTC m=+53.912974015" watchObservedRunningTime="2026-07-24 18:45:15.522705375 +0000 UTC m=+53.913656851"
Jul 24 18:45:15.538507 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:15.538465    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9k9h7" podStartSLOduration=9.374962811 podStartE2EDuration="27.538443924s" totalImagesPullingTime="18.163481113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:44:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:45:15.53779677 +0000 UTC m=+53.928748248" watchObservedRunningTime="2026-07-24 18:45:15.538443924 +0000 UTC m=+53.929395403"
Jul 24 18:45:16.198846 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.198771    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:16.198998 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:16.198906    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:16.343568 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.343537    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9k9h7"
Jul 24 18:45:16.344294 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.344260    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9k9h7"
Jul 24 18:45:16.426106 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.426075    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 18:45:16.426713 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.426538    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-889p5" event={"ID":"e5a77bc5-eeda-4e10-872e-190ad4ffeff5","Type":"ContainerStarted","Data":"1b6f1219a903d39823048597e13804c4ecda2bf8422c247b424193aaf57432f8"}
Jul 24 18:45:16.426713 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.426605    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-889p5" event={"ID":"e5a77bc5-eeda-4e10-872e-190ad4ffeff5","Type":"ContainerStarted","Data":"81c13f29e2ae29e9c8cac43bf6fa12cb4e55ddf41fba2772df7fc20ac4a9fb20"}
Jul 24 18:45:16.428378 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.428351    2563 generic.go:356] "Generic (PLEG): container finished" podID="076e1bdd-08e1-4c23-8029-91474ae2132a" containerID="cbee883a6e6c78db0c26965538306766ca53fbe554ed162345658a88579abbef" exitCode=0
Jul 24 18:45:16.428545 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.428444    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cxgj5" event={"ID":"076e1bdd-08e1-4c23-8029-91474ae2132a","Type":"ContainerDied","Data":"cbee883a6e6c78db0c26965538306766ca53fbe554ed162345658a88579abbef"}
Jul 24 18:45:16.430577 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.430546    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" event={"ID":"49c209b3e6e7d8404ad2a977c0bb87c9","Type":"ContainerStarted","Data":"55649329ebcd1949d47d92dd2951fed3e7b9925e69e9e7311dc60c2720f27310"}
Jul 24 18:45:16.432317 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.432289    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dtpsd" event={"ID":"33101463-3e59-49a7-91d4-947558c4b6c7","Type":"ContainerStarted","Data":"d4d5b1e36b7735664d1eaa462447122d7364032b64ec21eaa7d29a1aabc9de57"}
Jul 24 18:45:16.433100 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.433075    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9k9h7"
Jul 24 18:45:16.433747 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.433729    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9k9h7"
Jul 24 18:45:16.461619 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.461578    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" podStartSLOduration=21.461564745 podStartE2EDuration="21.461564745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:45:16.461093916 +0000 UTC m=+54.852045393" watchObservedRunningTime="2026-07-24 18:45:16.461564745 +0000 UTC m=+54.852516221"
Jul 24 18:45:16.493145 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.493094    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dtpsd" podStartSLOduration=9.319100486 podStartE2EDuration="27.49308105s" totalImagesPullingTime="18.173980564s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:44:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:45:16.492693247 +0000 UTC m=+54.883644727" watchObservedRunningTime="2026-07-24 18:45:16.49308105 +0000 UTC m=+54.884032527"
Jul 24 18:45:16.784556 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:16.784515    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 18:45:17.115575 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:17.115445    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T18:45:16.784552098Z","UUID":"5ce3fdc9-7869-4b86-88b3-2fee67c60bac","Handler":null,"Name":"","Endpoint":""}
Jul 24 18:45:17.117675 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:17.117639    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 18:45:17.117838 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:17.117687    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 18:45:17.198947 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:17.198913    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:17.198947 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:17.198944    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:17.199222 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:17.199011    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:17.199222 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:17.199163    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:17.437571 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:17.437497    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cxgj5" event={"ID":"076e1bdd-08e1-4c23-8029-91474ae2132a","Type":"ContainerStarted","Data":"cc1bef155a664c312c9f51c598bcef2ca1c09caf0f5bcd5910d9063c0a3caf25"}
Jul 24 18:45:17.438064 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:17.437572    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cxgj5" event={"ID":"076e1bdd-08e1-4c23-8029-91474ae2132a","Type":"ContainerStarted","Data":"00b24363a1b7f42e17b055922bbd969d296355e15fc5b21306009c2632a8582d"}
Jul 24 18:45:17.439941 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:17.439913    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb" event={"ID":"24465b9d-ca62-4506-933b-f0b4096ca5da","Type":"ContainerStarted","Data":"ee67159cbde018d0add157f0958b7d47d082161f9b6d3103f90955d3d74bb3e9"}
Jul 24 18:45:17.456083 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:17.456027    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cxgj5" podStartSLOduration=10.113202362 podStartE2EDuration="11.456008442s" totalImagesPullingTime="1.187315032s" totalInitContainerRuntime="155.491048ms" isStatefulPod=true podCreationTimestamp="2026-07-24 18:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:45:17.454866005 +0000 UTC m=+55.845817482" watchObservedRunningTime="2026-07-24 18:45:17.456008442 +0000 UTC m=+55.846959922"
Jul 24 18:45:18.199353 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:18.198918    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:18.199525 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:18.199315    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:18.446443 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:18.446372    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb" event={"ID":"24465b9d-ca62-4506-933b-f0b4096ca5da","Type":"ContainerStarted","Data":"ad7b67cff9f846ef2f241f080bc159f7017439166e6ee0990747afee6bccbdde"}
Jul 24 18:45:18.450134 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:18.450068    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 18:45:18.450557 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:18.450529    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-889p5" event={"ID":"e5a77bc5-eeda-4e10-872e-190ad4ffeff5","Type":"ContainerStarted","Data":"9c4ae6e8b8017a60ab026b9448f41484d2f6c5bcf5337df7259d5fe6c5e64e11"}
Jul 24 18:45:19.198848 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:19.198809    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:19.199063 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:19.198811    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:19.199063 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:19.198948    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:19.199063 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:19.199026    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:20.198148 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:20.198123    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:20.198683 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:20.198251    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:20.458792 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:20.458697    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 18:45:20.459157 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:20.459117    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-889p5" event={"ID":"e5a77bc5-eeda-4e10-872e-190ad4ffeff5","Type":"ContainerStarted","Data":"dca3f521b4510d5bce08d352f42f820428dabec71c694e141c62c41f53ac3417"}
Jul 24 18:45:20.459570 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:20.459412    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:45:20.459570 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:20.459484    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:45:20.459733 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:20.459667    2563 scope.go:117] "RemoveContainer" containerID="880591db7c87ccf3b487948bea669b3883bf2167635dd26933c876f943f25433"
Jul 24 18:45:20.463263 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:20.463240    2563 generic.go:356] "Generic (PLEG): container finished" podID="5eda199b-9a46-4bdd-919e-dbacd4631428" containerID="7bbd7cd8f5bb8093368ace9bd8af85d8dfdf81aba0d9d5ea6abb604beb85457d" exitCode=0
Jul 24 18:45:20.463408 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:20.463280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpk4s" event={"ID":"5eda199b-9a46-4bdd-919e-dbacd4631428","Type":"ContainerDied","Data":"7bbd7cd8f5bb8093368ace9bd8af85d8dfdf81aba0d9d5ea6abb604beb85457d"}
Jul 24 18:45:20.477888 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:20.477867    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:45:20.511521 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:20.511353    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gglrb" podStartSLOduration=10.277986475 podStartE2EDuration="31.511340164s" totalImagesPullingTime="21.233353689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:44:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:45:18.468382189 +0000 UTC m=+56.859333691" watchObservedRunningTime="2026-07-24 18:45:20.511340164 +0000 UTC m=+58.902291641"
Jul 24 18:45:21.198541 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:21.198511    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:21.199098 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:21.198636    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:21.199098 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:21.198696    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:21.199098 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:21.198802    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:21.470574 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:21.470505    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 18:45:21.470825 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:21.470804    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-889p5" event={"ID":"e5a77bc5-eeda-4e10-872e-190ad4ffeff5","Type":"ContainerStarted","Data":"3a7436b1e4f1b0e1ee8f5c0ce71c2872e0925f82326194e7039b166a22184b98"}
Jul 24 18:45:21.470998 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:21.470979    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:45:21.486343 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:21.486322    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:45:21.539219 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:21.537152    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-889p5" podStartSLOduration=14.049029337 podStartE2EDuration="32.537134953s" totalImagesPullingTime="18.488105616s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:44:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:45:21.50351455 +0000 UTC m=+59.894466026" watchObservedRunningTime="2026-07-24 18:45:21.537134953 +0000 UTC m=+59.928086432"
Jul 24 18:45:22.199145 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:22.199108    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:22.199878 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:22.199214    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:22.477594 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:22.477323    2563 generic.go:356] "Generic (PLEG): container finished" podID="5eda199b-9a46-4bdd-919e-dbacd4631428" containerID="f26d34d7c064d947248bde24c7418e641dfb0108d6fdc515cb0614cb72671553" exitCode=0
Jul 24 18:45:22.477594 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:22.477419    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpk4s" event={"ID":"5eda199b-9a46-4bdd-919e-dbacd4631428","Type":"ContainerDied","Data":"f26d34d7c064d947248bde24c7418e641dfb0108d6fdc515cb0614cb72671553"}
Jul 24 18:45:22.521243 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:22.521214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jwsd9"]
Jul 24 18:45:22.521509 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:22.521334    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:22.521509 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:22.521475    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:22.521985 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:22.521965    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-psbp5"]
Jul 24 18:45:22.522082 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:22.522059    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:22.522178 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:22.522155    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:22.522768 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:22.522741    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zgg7r"]
Jul 24 18:45:22.522865 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:22.522837    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:22.522940 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:22.522919    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:24.198893 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:24.198856    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:24.199270 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:24.198864    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:24.199270 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:24.198979    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:24.199270 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:24.199002    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:24.199270 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:24.199080    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:24.199270 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:24.199187    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:24.484531 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:24.484446    2563 generic.go:356] "Generic (PLEG): container finished" podID="5eda199b-9a46-4bdd-919e-dbacd4631428" containerID="ffa72e81e18a7586ce254d8a21ee5e0603f930e5b7e4ab46f1007fee90124e47" exitCode=0
Jul 24 18:45:24.484531 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:24.484516    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpk4s" event={"ID":"5eda199b-9a46-4bdd-919e-dbacd4631428","Type":"ContainerDied","Data":"ffa72e81e18a7586ce254d8a21ee5e0603f930e5b7e4ab46f1007fee90124e47"}
Jul 24 18:45:26.198322 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:26.198285    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:26.198849 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:26.198289    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:26.198849 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:26.198462    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:26.198849 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:26.198520    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:26.198849 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:26.198285    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:26.198849 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:26.198602    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:27.979315 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:27.979284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:27.979828 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:27.979325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs\") pod \"network-metrics-daemon-zgg7r\" (UID: \"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19\") " pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:27.979828 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:27.979453    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 18:45:27.979828 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:27.979509    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret podName:c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3 nodeName:}" failed. No retries permitted until 2026-07-24 18:45:59.979492085 +0000 UTC m=+98.370443540 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret") pod "global-pull-secret-syncer-jwsd9" (UID: "c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3") : object "kube-system"/"original-pull-secret" not registered
Jul 24 18:45:27.979828 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:27.979530    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 18:45:27.979828 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:27.979603    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs podName:7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19 nodeName:}" failed. No retries permitted until 2026-07-24 18:45:59.979583591 +0000 UTC m=+98.370535051 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs") pod "network-metrics-daemon-zgg7r" (UID: "7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 18:45:28.080734 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.080699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx8hz\" (UniqueName: \"kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz\") pod \"network-check-target-psbp5\" (UID: \"67e048f7-9b14-45bd-860a-8cc46c8ecd19\") " pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:28.080916 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:28.080893    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 18:45:28.080977 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:28.080929    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 18:45:28.080977 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:28.080943    2563 projected.go:196] Error preparing data for projected volume kube-api-access-vx8hz for pod openshift-network-diagnostics/network-check-target-psbp5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 18:45:28.081082 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:28.081004    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz podName:67e048f7-9b14-45bd-860a-8cc46c8ecd19 nodeName:}" failed. No retries permitted until 2026-07-24 18:46:00.080989723 +0000 UTC m=+98.471941178 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx8hz" (UniqueName: "kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz") pod "network-check-target-psbp5" (UID: "67e048f7-9b14-45bd-860a-8cc46c8ecd19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 18:45:28.198317 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.198291    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:28.198517 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.198354    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:28.198517 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.198357    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:28.198517 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:28.198475    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psbp5" podUID="67e048f7-9b14-45bd-860a-8cc46c8ecd19"
Jul 24 18:45:28.198699 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:28.198565    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zgg7r" podUID="7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19"
Jul 24 18:45:28.198699 ip-10-0-141-213 kubenswrapper[2563]: E0724 18:45:28.198673    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jwsd9" podUID="c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3"
Jul 24 18:45:28.367435 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.367406    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeReady"
Jul 24 18:45:28.367604 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.367587    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 18:45:28.495033 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.495007    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nrlxk"]
Jul 24 18:45:28.496982 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.496948    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:28.499532 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.499508    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 18:45:28.499532 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.499531    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 18:45:28.500917 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.500892    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fjp2z\""
Jul 24 18:45:28.505513 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.505494    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fljrs"]
Jul 24 18:45:28.507028 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.507010    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fljrs"
Jul 24 18:45:28.511589 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.511564    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-twnl6\""
Jul 24 18:45:28.511688 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.511641    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 18:45:28.511942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.511912    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 18:45:28.512262 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.512242    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 18:45:28.516088 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.516066    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nrlxk"]
Jul 24 18:45:28.532424 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.532377    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fljrs"]
Jul 24 18:45:28.584996 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.584910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7gnh\" (UniqueName: \"kubernetes.io/projected/e65a79e4-e43c-43a5-8a73-bff5b0a4d8de-kube-api-access-r7gnh\") pod \"dns-default-nrlxk\" (UID: \"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de\") " pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:28.584996 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.584975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e65a79e4-e43c-43a5-8a73-bff5b0a4d8de-metrics-tls\") pod \"dns-default-nrlxk\" (UID: \"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de\") " pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:28.585187 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.585043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rngq8\" (UniqueName: \"kubernetes.io/projected/a3e1f504-4766-4c99-b3b1-670a846755d7-kube-api-access-rngq8\") pod \"ingress-canary-fljrs\" (UID: \"a3e1f504-4766-4c99-b3b1-670a846755d7\") " pod="openshift-ingress-canary/ingress-canary-fljrs"
Jul 24 18:45:28.585187 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.585118    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e65a79e4-e43c-43a5-8a73-bff5b0a4d8de-tmp-dir\") pod \"dns-default-nrlxk\" (UID: \"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de\") " pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:28.585294 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.585184    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e65a79e4-e43c-43a5-8a73-bff5b0a4d8de-config-volume\") pod \"dns-default-nrlxk\" (UID: \"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de\") " pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:28.585294 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.585215    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3e1f504-4766-4c99-b3b1-670a846755d7-cert\") pod \"ingress-canary-fljrs\" (UID: \"a3e1f504-4766-4c99-b3b1-670a846755d7\") " pod="openshift-ingress-canary/ingress-canary-fljrs"
Jul 24 18:45:28.587562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.587532    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mnpws"]
Jul 24 18:45:28.590079 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.590053    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.592774 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.592749    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 18:45:28.592911 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.592780    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 18:45:28.592976 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.592916    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 18:45:28.593030 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.592999    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5dvls\""
Jul 24 18:45:28.593136 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.593115    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 18:45:28.602233 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.602209    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mnpws"]
Jul 24 18:45:28.685949 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.685912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e65a79e4-e43c-43a5-8a73-bff5b0a4d8de-config-volume\") pod \"dns-default-nrlxk\" (UID: \"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de\") " pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:28.685949 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.685954    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3e1f504-4766-4c99-b3b1-670a846755d7-cert\") pod \"ingress-canary-fljrs\" (UID: \"a3e1f504-4766-4c99-b3b1-670a846755d7\") " pod="openshift-ingress-canary/ingress-canary-fljrs"
Jul 24 18:45:28.686175 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.685987    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a81134d-1a8b-44e8-8d59-0346a25f0366-data-volume\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.686175 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.686039    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a81134d-1a8b-44e8-8d59-0346a25f0366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.686175 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.686068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7gnh\" (UniqueName: \"kubernetes.io/projected/e65a79e4-e43c-43a5-8a73-bff5b0a4d8de-kube-api-access-r7gnh\") pod \"dns-default-nrlxk\" (UID: \"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de\") " pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:28.686175 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.686098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a81134d-1a8b-44e8-8d59-0346a25f0366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.686175 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.686149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9mhk\" (UniqueName: \"kubernetes.io/projected/1a81134d-1a8b-44e8-8d59-0346a25f0366-kube-api-access-f9mhk\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.686409 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.686191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a81134d-1a8b-44e8-8d59-0346a25f0366-crio-socket\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.686409 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.686224    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e65a79e4-e43c-43a5-8a73-bff5b0a4d8de-metrics-tls\") pod \"dns-default-nrlxk\" (UID: \"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de\") " pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:28.686409 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.686250    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rngq8\" (UniqueName: \"kubernetes.io/projected/a3e1f504-4766-4c99-b3b1-670a846755d7-kube-api-access-rngq8\") pod \"ingress-canary-fljrs\" (UID: \"a3e1f504-4766-4c99-b3b1-670a846755d7\") " pod="openshift-ingress-canary/ingress-canary-fljrs"
Jul 24 18:45:28.686409 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.686300    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e65a79e4-e43c-43a5-8a73-bff5b0a4d8de-tmp-dir\") pod \"dns-default-nrlxk\" (UID: \"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de\") " pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:28.686609 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.686567    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e65a79e4-e43c-43a5-8a73-bff5b0a4d8de-config-volume\") pod \"dns-default-nrlxk\" (UID: \"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de\") " pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:28.686699 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.686671    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e65a79e4-e43c-43a5-8a73-bff5b0a4d8de-tmp-dir\") pod \"dns-default-nrlxk\" (UID: \"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de\") " pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:28.690356 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.690232    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e65a79e4-e43c-43a5-8a73-bff5b0a4d8de-metrics-tls\") pod \"dns-default-nrlxk\" (UID: \"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de\") " pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:28.690477 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.690460    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3e1f504-4766-4c99-b3b1-670a846755d7-cert\") pod \"ingress-canary-fljrs\" (UID: \"a3e1f504-4766-4c99-b3b1-670a846755d7\") " pod="openshift-ingress-canary/ingress-canary-fljrs"
Jul 24 18:45:28.694107 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.694075    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7gnh\" (UniqueName: \"kubernetes.io/projected/e65a79e4-e43c-43a5-8a73-bff5b0a4d8de-kube-api-access-r7gnh\") pod \"dns-default-nrlxk\" (UID: \"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de\") " pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:28.694270 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.694250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rngq8\" (UniqueName: \"kubernetes.io/projected/a3e1f504-4766-4c99-b3b1-670a846755d7-kube-api-access-rngq8\") pod \"ingress-canary-fljrs\" (UID: \"a3e1f504-4766-4c99-b3b1-670a846755d7\") " pod="openshift-ingress-canary/ingress-canary-fljrs"
Jul 24 18:45:28.787638 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.787605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a81134d-1a8b-44e8-8d59-0346a25f0366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.787773 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.787663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a81134d-1a8b-44e8-8d59-0346a25f0366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.787773 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.787691    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9mhk\" (UniqueName: \"kubernetes.io/projected/1a81134d-1a8b-44e8-8d59-0346a25f0366-kube-api-access-f9mhk\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.787773 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.787726    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a81134d-1a8b-44e8-8d59-0346a25f0366-crio-socket\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.787893 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.787812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a81134d-1a8b-44e8-8d59-0346a25f0366-data-volume\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.788123 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.788093    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a81134d-1a8b-44e8-8d59-0346a25f0366-data-volume\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.788176 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.788149    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a81134d-1a8b-44e8-8d59-0346a25f0366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.788343 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.788323    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a81134d-1a8b-44e8-8d59-0346a25f0366-crio-socket\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.790228 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.790206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a81134d-1a8b-44e8-8d59-0346a25f0366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.796228 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.796205    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9mhk\" (UniqueName: \"kubernetes.io/projected/1a81134d-1a8b-44e8-8d59-0346a25f0366-kube-api-access-f9mhk\") pod \"insights-runtime-extractor-mnpws\" (UID: \"1a81134d-1a8b-44e8-8d59-0346a25f0366\") " pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.807004 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.806983    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:28.815824 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.815805    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fljrs"
Jul 24 18:45:28.900300 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.900252    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mnpws"
Jul 24 18:45:28.963136 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.963106    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fljrs"]
Jul 24 18:45:28.965415 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:28.965363    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nrlxk"]
Jul 24 18:45:28.968649 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:45:28.968620    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode65a79e4_e43c_43a5_8a73_bff5b0a4d8de.slice/crio-d96d547f6e9fcbac80af5fc5b1bec088df73e8ec7d53d21a87cc2e9ba1c54e5c WatchSource:0}: Error finding container d96d547f6e9fcbac80af5fc5b1bec088df73e8ec7d53d21a87cc2e9ba1c54e5c: Status 404 returned error can't find the container with id d96d547f6e9fcbac80af5fc5b1bec088df73e8ec7d53d21a87cc2e9ba1c54e5c
Jul 24 18:45:28.969434 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:45:28.969412    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e1f504_4766_4c99_b3b1_670a846755d7.slice/crio-790aa01ff3df57f7eb00c778b7d65b5bf755e8ad1ef2e7c3c2bed9bf43fb46f9 WatchSource:0}: Error finding container 790aa01ff3df57f7eb00c778b7d65b5bf755e8ad1ef2e7c3c2bed9bf43fb46f9: Status 404 returned error can't find the container with id 790aa01ff3df57f7eb00c778b7d65b5bf755e8ad1ef2e7c3c2bed9bf43fb46f9
Jul 24 18:45:29.081021 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:29.080845    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mnpws"]
Jul 24 18:45:29.084234 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:45:29.084204    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a81134d_1a8b_44e8_8d59_0346a25f0366.slice/crio-905ff2d0791425a4c765bd6868d2fab0f0cd56600686ea425575b7a0435d16da WatchSource:0}: Error finding container 905ff2d0791425a4c765bd6868d2fab0f0cd56600686ea425575b7a0435d16da: Status 404 returned error can't find the container with id 905ff2d0791425a4c765bd6868d2fab0f0cd56600686ea425575b7a0435d16da
Jul 24 18:45:29.502440 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:29.502374    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nrlxk" event={"ID":"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de","Type":"ContainerStarted","Data":"d96d547f6e9fcbac80af5fc5b1bec088df73e8ec7d53d21a87cc2e9ba1c54e5c"}
Jul 24 18:45:29.504726 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:29.504695    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fljrs" event={"ID":"a3e1f504-4766-4c99-b3b1-670a846755d7","Type":"ContainerStarted","Data":"790aa01ff3df57f7eb00c778b7d65b5bf755e8ad1ef2e7c3c2bed9bf43fb46f9"}
Jul 24 18:45:29.506934 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:29.506852    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mnpws" event={"ID":"1a81134d-1a8b-44e8-8d59-0346a25f0366","Type":"ContainerStarted","Data":"905ff2d0791425a4c765bd6868d2fab0f0cd56600686ea425575b7a0435d16da"}
Jul 24 18:45:29.513241 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:29.512346    2563 generic.go:356] "Generic (PLEG): container finished" podID="5eda199b-9a46-4bdd-919e-dbacd4631428" containerID="6227be88dd55205293315a8aeb85a709cd9ecf8f2cd5379210fef7dc4851f207" exitCode=0
Jul 24 18:45:29.513241 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:29.512383    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpk4s" event={"ID":"5eda199b-9a46-4bdd-919e-dbacd4631428","Type":"ContainerDied","Data":"6227be88dd55205293315a8aeb85a709cd9ecf8f2cd5379210fef7dc4851f207"}
Jul 24 18:45:30.199370 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:30.198198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:45:30.199370 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:30.199123    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:45:30.200355 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:30.199961    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:45:30.203599 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:30.201100    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 18:45:30.203599 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:30.201436    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 18:45:30.203599 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:30.201654    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6w7xn\""
Jul 24 18:45:30.203599 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:30.201720    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 18:45:30.203599 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:30.202056    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 18:45:30.203599 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:30.202101    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-96zrd\""
Jul 24 18:45:30.521044 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:30.520967    2563 generic.go:356] "Generic (PLEG): container finished" podID="5eda199b-9a46-4bdd-919e-dbacd4631428" containerID="d17f5893b37f6ad2f02b73cb02ac4628e50c4096847f3bab826177dd4138708c" exitCode=0
Jul 24 18:45:30.521044 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:30.521029    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpk4s" event={"ID":"5eda199b-9a46-4bdd-919e-dbacd4631428","Type":"ContainerDied","Data":"d17f5893b37f6ad2f02b73cb02ac4628e50c4096847f3bab826177dd4138708c"}
Jul 24 18:45:31.528893 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:31.528812    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpk4s" event={"ID":"5eda199b-9a46-4bdd-919e-dbacd4631428","Type":"ContainerStarted","Data":"ef1ef884924cbfb2002bb2d16f950078fc3008c65fab64ef1cbd8cb44f00a110"}
Jul 24 18:45:31.553530 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:31.553474    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tpk4s" podStartSLOduration=13.292773793 podStartE2EDuration="42.553456688s" totalImagesPullingTime="29.260682895s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:44:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:45:31.553213347 +0000 UTC m=+69.944164825" watchObservedRunningTime="2026-07-24 18:45:31.553456688 +0000 UTC m=+69.944408158"
Jul 24 18:45:36.545280 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:36.545233    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nrlxk" event={"ID":"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de","Type":"ContainerStarted","Data":"87627c4e35cd10113de5c4a71881dda10744d500c31801dd696da20ffbfcba8c"}
Jul 24 18:45:36.545721 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:36.545316    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nrlxk" event={"ID":"e65a79e4-e43c-43a5-8a73-bff5b0a4d8de","Type":"ContainerStarted","Data":"3aff96a7f5f8a8aab6ad2059c689f478054713cd134f821f3c171ffd40da5b06"}
Jul 24 18:45:36.545721 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:36.545467    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:36.546773 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:36.546745    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fljrs" event={"ID":"a3e1f504-4766-4c99-b3b1-670a846755d7","Type":"ContainerStarted","Data":"f7dc9130e2e588c8c30c2f33eb08a35e23883b85bd23fccabe2ac9c68d43f801"}
Jul 24 18:45:36.548307 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:36.548284    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mnpws" event={"ID":"1a81134d-1a8b-44e8-8d59-0346a25f0366","Type":"ContainerStarted","Data":"0b14f8e24b366ba0b8709641d332c27cde24d584275fd708acab28db32922e5d"}
Jul 24 18:45:36.580911 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:36.580859    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nrlxk" podStartSLOduration=1.975606263 podStartE2EDuration="8.580841442s" totalImagesPullingTime="6.605235179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:45:36.579945551 +0000 UTC m=+74.970897028" watchObservedRunningTime="2026-07-24 18:45:36.580841442 +0000 UTC m=+74.971792921"
Jul 24 18:45:36.598126 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:36.598077    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fljrs" podStartSLOduration=1.9934923530000002 podStartE2EDuration="8.598060212s" totalImagesPullingTime="6.604567859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:45:36.597293649 +0000 UTC m=+74.988245128" watchObservedRunningTime="2026-07-24 18:45:36.598060212 +0000 UTC m=+74.989011692"
Jul 24 18:45:39.559365 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:39.559332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mnpws" event={"ID":"1a81134d-1a8b-44e8-8d59-0346a25f0366","Type":"ContainerStarted","Data":"bc28ca34aa2670a274380e69f421a71d6a1f29ffdad530289ea2656a0c4b1455"}
Jul 24 18:45:43.576179 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:43.576143    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mnpws" event={"ID":"1a81134d-1a8b-44e8-8d59-0346a25f0366","Type":"ContainerStarted","Data":"bef898de39cb870bca38518677ce7328ecdf8886245a6a328ea1cc764ee5e433"}
Jul 24 18:45:43.603775 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:43.603727    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mnpws" podStartSLOduration=1.527351006 podStartE2EDuration="15.60371068s" totalImagesPullingTime="14.076359674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:45:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:45:43.602740477 +0000 UTC m=+81.993691955" watchObservedRunningTime="2026-07-24 18:45:43.60371068 +0000 UTC m=+81.994662158"
Jul 24 18:45:46.553881 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:46.553854    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nrlxk"
Jul 24 18:45:53.491684 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:45:53.491651    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-889p5"
Jul 24 18:46:00.029839 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.029798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:46:00.029839 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.029843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs\") pod \"network-metrics-daemon-zgg7r\" (UID: \"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19\") " pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:46:00.033682 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.033662    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19-metrics-certs\") pod \"network-metrics-daemon-zgg7r\" (UID: \"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19\") " pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:46:00.033740 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.033675    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3-original-pull-secret\") pod \"global-pull-secret-syncer-jwsd9\" (UID: \"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3\") " pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:46:00.130356 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.130318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx8hz\" (UniqueName: \"kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz\") pod \"network-check-target-psbp5\" (UID: \"67e048f7-9b14-45bd-860a-8cc46c8ecd19\") " pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:46:00.133041 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.133014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx8hz\" (UniqueName: \"kubernetes.io/projected/67e048f7-9b14-45bd-860a-8cc46c8ecd19-kube-api-access-vx8hz\") pod \"network-check-target-psbp5\" (UID: \"67e048f7-9b14-45bd-860a-8cc46c8ecd19\") " pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:46:00.214490 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.214456    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:46:00.225571 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.225542    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jwsd9"
Jul 24 18:46:00.232423 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.232369    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zgg7r"
Jul 24 18:46:00.356714 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.356680    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-psbp5"]
Jul 24 18:46:00.360115 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:46:00.360089    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e048f7_9b14_45bd_860a_8cc46c8ecd19.slice/crio-2d7a5f37ab0a377b2ad6b474d78dc381acb0ebfcc097e468d712af5afba939c2 WatchSource:0}: Error finding container 2d7a5f37ab0a377b2ad6b474d78dc381acb0ebfcc097e468d712af5afba939c2: Status 404 returned error can't find the container with id 2d7a5f37ab0a377b2ad6b474d78dc381acb0ebfcc097e468d712af5afba939c2
Jul 24 18:46:00.581964 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.581933    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jwsd9"]
Jul 24 18:46:00.585156 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.585127    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zgg7r"]
Jul 24 18:46:00.585294 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:46:00.585207    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc844f7c8_ddea_4daa_9d99_b0f2b54ab7e3.slice/crio-8ab9bfa08fd1df74950ea342f2b4e14b54829106fde41c099434879d632693d8 WatchSource:0}: Error finding container 8ab9bfa08fd1df74950ea342f2b4e14b54829106fde41c099434879d632693d8: Status 404 returned error can't find the container with id 8ab9bfa08fd1df74950ea342f2b4e14b54829106fde41c099434879d632693d8
Jul 24 18:46:00.589131 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:46:00.589110    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c5cdbc6_aede_4ffe_8fec_e5bd6c2ced19.slice/crio-979203c6a084c5f70d6e831d351df2956af7491dacc5230ea045321a3bf9f9ec WatchSource:0}: Error finding container 979203c6a084c5f70d6e831d351df2956af7491dacc5230ea045321a3bf9f9ec: Status 404 returned error can't find the container with id 979203c6a084c5f70d6e831d351df2956af7491dacc5230ea045321a3bf9f9ec
Jul 24 18:46:00.628773 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.628739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zgg7r" event={"ID":"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19","Type":"ContainerStarted","Data":"979203c6a084c5f70d6e831d351df2956af7491dacc5230ea045321a3bf9f9ec"}
Jul 24 18:46:00.629830 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.629796    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jwsd9" event={"ID":"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3","Type":"ContainerStarted","Data":"8ab9bfa08fd1df74950ea342f2b4e14b54829106fde41c099434879d632693d8"}
Jul 24 18:46:00.630824 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:00.630803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-psbp5" event={"ID":"67e048f7-9b14-45bd-860a-8cc46c8ecd19","Type":"ContainerStarted","Data":"2d7a5f37ab0a377b2ad6b474d78dc381acb0ebfcc097e468d712af5afba939c2"}
Jul 24 18:46:02.641128 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:02.641079    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zgg7r" event={"ID":"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19","Type":"ContainerStarted","Data":"9a003ef7bfb6c63ad9f8877557aeffb752c9099b72938c545ab2c3a7954a3b22"}
Jul 24 18:46:02.641768 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:02.641153    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zgg7r" event={"ID":"7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19","Type":"ContainerStarted","Data":"5f24f6f65766837225bd797812c8f7910fa6c096ec00d5bb7992a2c1470d0d07"}
Jul 24 18:46:02.657715 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:02.657657    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zgg7r" podStartSLOduration=72.493552629 podStartE2EDuration="1m13.657637163s" totalImagesPullingTime="1.164084534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:44:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:46:02.657432693 +0000 UTC m=+101.048384171" watchObservedRunningTime="2026-07-24 18:46:02.657637163 +0000 UTC m=+101.048588638"
Jul 24 18:46:03.647271 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:03.647237    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-psbp5" event={"ID":"67e048f7-9b14-45bd-860a-8cc46c8ecd19","Type":"ContainerStarted","Data":"17c0c3a6d74b78559771db21d1ad80f3379bfdf52e4405931ec8f3d68a95aeb0"}
Jul 24 18:46:03.647747 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:03.647324    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:46:03.663955 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:03.663897    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-psbp5" podStartSLOduration=71.593113027 podStartE2EDuration="1m14.663880596s" totalImagesPullingTime="3.070767569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:44:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:46:03.663727019 +0000 UTC m=+102.054678495" watchObservedRunningTime="2026-07-24 18:46:03.663880596 +0000 UTC m=+102.054832075"
Jul 24 18:46:05.656292 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:05.656259    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jwsd9" event={"ID":"c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3","Type":"ContainerStarted","Data":"a7c15eba7416c919b174e7a419006fdb369b1735d7dec7ba1ec1863d1fe8f643"}
Jul 24 18:46:05.672667 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:05.672615    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jwsd9" podStartSLOduration=72.00511296 podStartE2EDuration="1m16.672597259s" totalImagesPullingTime="4.667484299s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:44:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:46:05.671782554 +0000 UTC m=+104.062734044" watchObservedRunningTime="2026-07-24 18:46:05.672597259 +0000 UTC m=+104.063548735"
Jul 24 18:46:29.045849 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.045807    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 18:46:29.075090 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.075059    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 18:46:29.075233 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.075203    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.077730 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.077707    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 18:46:29.077991 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.077964    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 18:46:29.078098 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.077999    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 18:46:29.078098 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.077964    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 18:46:29.078098 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.078084    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 18:46:29.078258 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.078090    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8mvng\""
Jul 24 18:46:29.078258 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.078232    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 18:46:29.078520 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.078497    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 18:46:29.078606 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.078578    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 18:46:29.082591 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.082568    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 18:46:29.130811 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.130780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.130977 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.130818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.130977 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.130843    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37b0381f-1920-4032-95d3-e3f38ffc4149-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.130977 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.130871    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-config-volume\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.130977 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.130925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hhzd\" (UniqueName: \"kubernetes.io/projected/37b0381f-1920-4032-95d3-e3f38ffc4149-kube-api-access-7hhzd\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.130977 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.130961    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/37b0381f-1920-4032-95d3-e3f38ffc4149-config-out\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.131207 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.130985    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-web-config\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.131207 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.131025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37b0381f-1920-4032-95d3-e3f38ffc4149-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.131207 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.131087    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/37b0381f-1920-4032-95d3-e3f38ffc4149-tls-assets\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.131207 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.131109    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.131207 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.131129    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/37b0381f-1920-4032-95d3-e3f38ffc4149-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.131207 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.131149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.131207 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.131171    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.231831 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.231792    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.231974 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.231836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.232033 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.231972    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37b0381f-1920-4032-95d3-e3f38ffc4149-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.232033 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.232020    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-config-volume\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.232197 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.232052    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hhzd\" (UniqueName: \"kubernetes.io/projected/37b0381f-1920-4032-95d3-e3f38ffc4149-kube-api-access-7hhzd\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.232197 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.232083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/37b0381f-1920-4032-95d3-e3f38ffc4149-config-out\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.232197 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.232109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-web-config\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.232197 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.232145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37b0381f-1920-4032-95d3-e3f38ffc4149-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.232197 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.232177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/37b0381f-1920-4032-95d3-e3f38ffc4149-tls-assets\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.232476 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.232200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.232476 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.232233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/37b0381f-1920-4032-95d3-e3f38ffc4149-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.232476 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.232260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.232476 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.232297    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.233093 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.233064    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37b0381f-1920-4032-95d3-e3f38ffc4149-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.234029 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.234005    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37b0381f-1920-4032-95d3-e3f38ffc4149-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.234244 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.234219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/37b0381f-1920-4032-95d3-e3f38ffc4149-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.235651 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.235607    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.235651 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.235637    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.235791 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.235758    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.235836 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.235785    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.236000 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.235977    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-config-volume\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.236045 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.235993    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/37b0381f-1920-4032-95d3-e3f38ffc4149-config-out\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.236132 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.236116    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-web-config\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.237683 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.237626    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/37b0381f-1920-4032-95d3-e3f38ffc4149-tls-assets\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.237683 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.237646    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/37b0381f-1920-4032-95d3-e3f38ffc4149-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.241901 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.240793    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hhzd\" (UniqueName: \"kubernetes.io/projected/37b0381f-1920-4032-95d3-e3f38ffc4149-kube-api-access-7hhzd\") pod \"alertmanager-main-0\" (UID: \"37b0381f-1920-4032-95d3-e3f38ffc4149\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.384733 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.384697    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 18:46:29.525484 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.525453    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 18:46:29.529287 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:46:29.529258    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37b0381f_1920_4032_95d3_e3f38ffc4149.slice/crio-82d8e278bc141b3e83330ebd2707e5e7cdad9a8407f2ca23dba9ce6dab545da7 WatchSource:0}: Error finding container 82d8e278bc141b3e83330ebd2707e5e7cdad9a8407f2ca23dba9ce6dab545da7: Status 404 returned error can't find the container with id 82d8e278bc141b3e83330ebd2707e5e7cdad9a8407f2ca23dba9ce6dab545da7
Jul 24 18:46:29.731769 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:29.731681    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37b0381f-1920-4032-95d3-e3f38ffc4149","Type":"ContainerStarted","Data":"82d8e278bc141b3e83330ebd2707e5e7cdad9a8407f2ca23dba9ce6dab545da7"}
Jul 24 18:46:31.739745 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:31.739710    2563 generic.go:356] "Generic (PLEG): container finished" podID="37b0381f-1920-4032-95d3-e3f38ffc4149" containerID="0ddf238072c1421043ce150157c8b9e9833af53157ff5005c4e1e8057e53462d" exitCode=0
Jul 24 18:46:31.740196 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:31.739798    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37b0381f-1920-4032-95d3-e3f38ffc4149","Type":"ContainerDied","Data":"0ddf238072c1421043ce150157c8b9e9833af53157ff5005c4e1e8057e53462d"}
Jul 24 18:46:33.749042 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:33.749008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37b0381f-1920-4032-95d3-e3f38ffc4149","Type":"ContainerStarted","Data":"9aa8b32ad21d2aaf605970e8c4f8f654298ad8d43baf6dd1a18af5754bfc3d29"}
Jul 24 18:46:33.749428 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:33.749097    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37b0381f-1920-4032-95d3-e3f38ffc4149","Type":"ContainerStarted","Data":"11253465f2c5703019e1ae09c4379debe5a0631215635351d42bf36b05ea94b5"}
Jul 24 18:46:33.749428 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:33.749125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37b0381f-1920-4032-95d3-e3f38ffc4149","Type":"ContainerStarted","Data":"b53f9d7ff8cd6f3cd7568f5f76f8ad49c417b3c68121d92edb932e58e67762af"}
Jul 24 18:46:33.749428 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:33.749150    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37b0381f-1920-4032-95d3-e3f38ffc4149","Type":"ContainerStarted","Data":"98a27d123366ba36a182169e012fb3594b3c934278c251169cf0f9e8efb8c80f"}
Jul 24 18:46:33.749428 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:33.749173    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37b0381f-1920-4032-95d3-e3f38ffc4149","Type":"ContainerStarted","Data":"d46a34c1f1d290813b6b18112233d120d9c164097c91747a26f93781ca308af8"}
Jul 24 18:46:34.654096 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:34.654063    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-psbp5"
Jul 24 18:46:34.755632 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:34.755588    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37b0381f-1920-4032-95d3-e3f38ffc4149","Type":"ContainerStarted","Data":"894a23980c307abeb73e389072aa0b576885401c71222395ca264eec49829664"}
Jul 24 18:46:34.799440 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:46:34.799362    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.496099639 podStartE2EDuration="5.799342153s" totalImagesPullingTime="3.286318259s" totalInitContainerRuntime="16.924255ms" isStatefulPod=false podCreationTimestamp="2026-07-24 18:46:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:46:34.799093326 +0000 UTC m=+133.190044804" watchObservedRunningTime="2026-07-24 18:46:34.799342153 +0000 UTC m=+133.190293630"
Jul 24 18:47:22.357229 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.357146    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"]
Jul 24 18:47:22.360218 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.360200    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"
Jul 24 18:47:22.363747 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.363726    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mkm8h\""
Jul 24 18:47:22.363945 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.363925    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 18:47:22.364619 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.364602    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 18:47:22.374890 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.374861    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"]
Jul 24 18:47:22.490422 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.490350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59deb6c0-170d-4019-8f62-496fdbc23071-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm\" (UID: \"59deb6c0-170d-4019-8f62-496fdbc23071\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"
Jul 24 18:47:22.490609 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.490435    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59deb6c0-170d-4019-8f62-496fdbc23071-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm\" (UID: \"59deb6c0-170d-4019-8f62-496fdbc23071\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"
Jul 24 18:47:22.490609 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.490469    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48rdq\" (UniqueName: \"kubernetes.io/projected/59deb6c0-170d-4019-8f62-496fdbc23071-kube-api-access-48rdq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm\" (UID: \"59deb6c0-170d-4019-8f62-496fdbc23071\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"
Jul 24 18:47:22.591597 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.591556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59deb6c0-170d-4019-8f62-496fdbc23071-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm\" (UID: \"59deb6c0-170d-4019-8f62-496fdbc23071\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"
Jul 24 18:47:22.591751 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.591615    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59deb6c0-170d-4019-8f62-496fdbc23071-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm\" (UID: \"59deb6c0-170d-4019-8f62-496fdbc23071\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"
Jul 24 18:47:22.591751 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.591645    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48rdq\" (UniqueName: \"kubernetes.io/projected/59deb6c0-170d-4019-8f62-496fdbc23071-kube-api-access-48rdq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm\" (UID: \"59deb6c0-170d-4019-8f62-496fdbc23071\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"
Jul 24 18:47:22.591965 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.591942    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59deb6c0-170d-4019-8f62-496fdbc23071-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm\" (UID: \"59deb6c0-170d-4019-8f62-496fdbc23071\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"
Jul 24 18:47:22.592022 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.591970    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59deb6c0-170d-4019-8f62-496fdbc23071-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm\" (UID: \"59deb6c0-170d-4019-8f62-496fdbc23071\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"
Jul 24 18:47:22.601319 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.601299    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48rdq\" (UniqueName: \"kubernetes.io/projected/59deb6c0-170d-4019-8f62-496fdbc23071-kube-api-access-48rdq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm\" (UID: \"59deb6c0-170d-4019-8f62-496fdbc23071\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"
Jul 24 18:47:22.669028 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.668927    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"
Jul 24 18:47:22.808640 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.808610    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"]
Jul 24 18:47:22.811958 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:47:22.811931    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59deb6c0_170d_4019_8f62_496fdbc23071.slice/crio-0ef1fd5ff64f5dbba35826749722e299f25ef3c4a3c5677e0127515f7c9bab7d WatchSource:0}: Error finding container 0ef1fd5ff64f5dbba35826749722e299f25ef3c4a3c5677e0127515f7c9bab7d: Status 404 returned error can't find the container with id 0ef1fd5ff64f5dbba35826749722e299f25ef3c4a3c5677e0127515f7c9bab7d
Jul 24 18:47:22.918526 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:22.918484    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm" event={"ID":"59deb6c0-170d-4019-8f62-496fdbc23071","Type":"ContainerStarted","Data":"0ef1fd5ff64f5dbba35826749722e299f25ef3c4a3c5677e0127515f7c9bab7d"}
Jul 24 18:47:29.944033 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:29.943998    2563 generic.go:356] "Generic (PLEG): container finished" podID="59deb6c0-170d-4019-8f62-496fdbc23071" containerID="78e5f85c9ecf08eb5bd3164a4219b4c33c625f6e12de97ccafdf73fbfa256475" exitCode=0
Jul 24 18:47:29.944033 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:29.944047    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm" event={"ID":"59deb6c0-170d-4019-8f62-496fdbc23071","Type":"ContainerDied","Data":"78e5f85c9ecf08eb5bd3164a4219b4c33c625f6e12de97ccafdf73fbfa256475"}
Jul 24 18:47:32.957084 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:32.957044    2563 generic.go:356] "Generic (PLEG): container finished" podID="59deb6c0-170d-4019-8f62-496fdbc23071" containerID="a62365e73cafd72b704324c18b2cbbb874eb1e5cea557e658a08b5e92cf83672" exitCode=0
Jul 24 18:47:32.957573 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:32.957100    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm" event={"ID":"59deb6c0-170d-4019-8f62-496fdbc23071","Type":"ContainerDied","Data":"a62365e73cafd72b704324c18b2cbbb874eb1e5cea557e658a08b5e92cf83672"}
Jul 24 18:47:39.985259 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:39.985221    2563 generic.go:356] "Generic (PLEG): container finished" podID="59deb6c0-170d-4019-8f62-496fdbc23071" containerID="1a3ad7262c7766162d7daef21675efb93a6b2ef45773144b98dc208b6a6fbf4b" exitCode=0
Jul 24 18:47:39.985751 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:39.985275    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm" event={"ID":"59deb6c0-170d-4019-8f62-496fdbc23071","Type":"ContainerDied","Data":"1a3ad7262c7766162d7daef21675efb93a6b2ef45773144b98dc208b6a6fbf4b"}
Jul 24 18:47:41.129344 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:41.129319    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"
Jul 24 18:47:41.230165 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:41.230126    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59deb6c0-170d-4019-8f62-496fdbc23071-util\") pod \"59deb6c0-170d-4019-8f62-496fdbc23071\" (UID: \"59deb6c0-170d-4019-8f62-496fdbc23071\") "
Jul 24 18:47:41.230361 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:41.230185    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59deb6c0-170d-4019-8f62-496fdbc23071-bundle\") pod \"59deb6c0-170d-4019-8f62-496fdbc23071\" (UID: \"59deb6c0-170d-4019-8f62-496fdbc23071\") "
Jul 24 18:47:41.230361 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:41.230214    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48rdq\" (UniqueName: \"kubernetes.io/projected/59deb6c0-170d-4019-8f62-496fdbc23071-kube-api-access-48rdq\") pod \"59deb6c0-170d-4019-8f62-496fdbc23071\" (UID: \"59deb6c0-170d-4019-8f62-496fdbc23071\") "
Jul 24 18:47:41.230849 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:41.230823    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59deb6c0-170d-4019-8f62-496fdbc23071-bundle" (OuterVolumeSpecName: "bundle") pod "59deb6c0-170d-4019-8f62-496fdbc23071" (UID: "59deb6c0-170d-4019-8f62-496fdbc23071"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:47:41.232699 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:41.232672    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59deb6c0-170d-4019-8f62-496fdbc23071-kube-api-access-48rdq" (OuterVolumeSpecName: "kube-api-access-48rdq") pod "59deb6c0-170d-4019-8f62-496fdbc23071" (UID: "59deb6c0-170d-4019-8f62-496fdbc23071"). InnerVolumeSpecName "kube-api-access-48rdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:47:41.234347 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:41.234315    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59deb6c0-170d-4019-8f62-496fdbc23071-util" (OuterVolumeSpecName: "util") pod "59deb6c0-170d-4019-8f62-496fdbc23071" (UID: "59deb6c0-170d-4019-8f62-496fdbc23071"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:47:41.330609 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:41.330580    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59deb6c0-170d-4019-8f62-496fdbc23071-bundle\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:47:41.330609 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:41.330607    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48rdq\" (UniqueName: \"kubernetes.io/projected/59deb6c0-170d-4019-8f62-496fdbc23071-kube-api-access-48rdq\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:47:41.330782 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:41.330619    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59deb6c0-170d-4019-8f62-496fdbc23071-util\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:47:41.994039 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:41.994006    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm" event={"ID":"59deb6c0-170d-4019-8f62-496fdbc23071","Type":"ContainerDied","Data":"0ef1fd5ff64f5dbba35826749722e299f25ef3c4a3c5677e0127515f7c9bab7d"}
Jul 24 18:47:41.994039 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:41.994028    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr9zm"
Jul 24 18:47:41.994235 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:41.994056    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ef1fd5ff64f5dbba35826749722e299f25ef3c4a3c5677e0127515f7c9bab7d"
Jul 24 18:47:45.456353 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.456318    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c6pxc"]
Jul 24 18:47:45.456746 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.456587    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59deb6c0-170d-4019-8f62-496fdbc23071" containerName="util"
Jul 24 18:47:45.456746 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.456598    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="59deb6c0-170d-4019-8f62-496fdbc23071" containerName="util"
Jul 24 18:47:45.456746 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.456610    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59deb6c0-170d-4019-8f62-496fdbc23071" containerName="extract"
Jul 24 18:47:45.456746 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.456616    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="59deb6c0-170d-4019-8f62-496fdbc23071" containerName="extract"
Jul 24 18:47:45.456746 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.456628    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59deb6c0-170d-4019-8f62-496fdbc23071" containerName="pull"
Jul 24 18:47:45.456746 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.456633    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="59deb6c0-170d-4019-8f62-496fdbc23071" containerName="pull"
Jul 24 18:47:45.456746 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.456668    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="59deb6c0-170d-4019-8f62-496fdbc23071" containerName="extract"
Jul 24 18:47:45.492730 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.492693    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c6pxc"]
Jul 24 18:47:45.492900 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.492815    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c6pxc"
Jul 24 18:47:45.495538 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.495512    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 24 18:47:45.495670 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.495649    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-qnmkv\""
Jul 24 18:47:45.495898 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.495878    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 24 18:47:45.557818 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.557774    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmjqv\" (UniqueName: \"kubernetes.io/projected/4f44fe06-3f0b-45da-a2f1-53167e6c5b47-kube-api-access-qmjqv\") pod \"cert-manager-operator-controller-manager-55c444bf86-c6pxc\" (UID: \"4f44fe06-3f0b-45da-a2f1-53167e6c5b47\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c6pxc"
Jul 24 18:47:45.558007 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.557841    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f44fe06-3f0b-45da-a2f1-53167e6c5b47-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-c6pxc\" (UID: \"4f44fe06-3f0b-45da-a2f1-53167e6c5b47\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c6pxc"
Jul 24 18:47:45.659105 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.659076    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f44fe06-3f0b-45da-a2f1-53167e6c5b47-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-c6pxc\" (UID: \"4f44fe06-3f0b-45da-a2f1-53167e6c5b47\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c6pxc"
Jul 24 18:47:45.659226 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.659140    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmjqv\" (UniqueName: \"kubernetes.io/projected/4f44fe06-3f0b-45da-a2f1-53167e6c5b47-kube-api-access-qmjqv\") pod \"cert-manager-operator-controller-manager-55c444bf86-c6pxc\" (UID: \"4f44fe06-3f0b-45da-a2f1-53167e6c5b47\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c6pxc"
Jul 24 18:47:45.659523 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.659504    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f44fe06-3f0b-45da-a2f1-53167e6c5b47-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-c6pxc\" (UID: \"4f44fe06-3f0b-45da-a2f1-53167e6c5b47\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c6pxc"
Jul 24 18:47:45.670743 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.670708    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmjqv\" (UniqueName: \"kubernetes.io/projected/4f44fe06-3f0b-45da-a2f1-53167e6c5b47-kube-api-access-qmjqv\") pod \"cert-manager-operator-controller-manager-55c444bf86-c6pxc\" (UID: \"4f44fe06-3f0b-45da-a2f1-53167e6c5b47\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c6pxc"
Jul 24 18:47:45.801493 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.801380    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c6pxc"
Jul 24 18:47:45.963722 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:45.963682    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c6pxc"]
Jul 24 18:47:45.967462 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:47:45.967426    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f44fe06_3f0b_45da_a2f1_53167e6c5b47.slice/crio-a21501b02c9a2785c4694910442c1e26a5ff2d5e1a4bf29528e7766c6f0de926 WatchSource:0}: Error finding container a21501b02c9a2785c4694910442c1e26a5ff2d5e1a4bf29528e7766c6f0de926: Status 404 returned error can't find the container with id a21501b02c9a2785c4694910442c1e26a5ff2d5e1a4bf29528e7766c6f0de926
Jul 24 18:47:46.009682 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:46.009650    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c6pxc" event={"ID":"4f44fe06-3f0b-45da-a2f1-53167e6c5b47","Type":"ContainerStarted","Data":"a21501b02c9a2785c4694910442c1e26a5ff2d5e1a4bf29528e7766c6f0de926"}
Jul 24 18:47:49.024078 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.024045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c6pxc" event={"ID":"4f44fe06-3f0b-45da-a2f1-53167e6c5b47","Type":"ContainerStarted","Data":"2e7064aec602f2eb4e380e53f6a73df5e9921343125b5d17f4a19e5419d5c6f3"}
Jul 24 18:47:49.046624 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.046526    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c6pxc" podStartSLOduration=1.737458612 podStartE2EDuration="4.046504078s" totalImagesPullingTime="2.309045466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:47:49.04526014 +0000 UTC m=+207.436211628" watchObservedRunningTime="2026-07-24 18:47:49.046504078 +0000 UTC m=+207.437455556"
Jul 24 18:47:49.651471 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.651438    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"]
Jul 24 18:47:49.656865 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.656831    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"
Jul 24 18:47:49.659638 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.659609    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 18:47:49.659821 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.659655    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 18:47:49.660043 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.660027    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mkm8h\""
Jul 24 18:47:49.674083 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.674056    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"]
Jul 24 18:47:49.693930 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.693900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gntfw\" (UniqueName: \"kubernetes.io/projected/d8b3eda9-d907-444d-bf57-1342e816791e-kube-api-access-gntfw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq\" (UID: \"d8b3eda9-d907-444d-bf57-1342e816791e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"
Jul 24 18:47:49.694064 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.693958    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d8b3eda9-d907-444d-bf57-1342e816791e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq\" (UID: \"d8b3eda9-d907-444d-bf57-1342e816791e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"
Jul 24 18:47:49.694064 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.694047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d8b3eda9-d907-444d-bf57-1342e816791e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq\" (UID: \"d8b3eda9-d907-444d-bf57-1342e816791e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"
Jul 24 18:47:49.794580 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.794545    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d8b3eda9-d907-444d-bf57-1342e816791e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq\" (UID: \"d8b3eda9-d907-444d-bf57-1342e816791e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"
Jul 24 18:47:49.794824 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.794600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d8b3eda9-d907-444d-bf57-1342e816791e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq\" (UID: \"d8b3eda9-d907-444d-bf57-1342e816791e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"
Jul 24 18:47:49.794824 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.794625    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gntfw\" (UniqueName: \"kubernetes.io/projected/d8b3eda9-d907-444d-bf57-1342e816791e-kube-api-access-gntfw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq\" (UID: \"d8b3eda9-d907-444d-bf57-1342e816791e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"
Jul 24 18:47:49.795022 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.794999    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d8b3eda9-d907-444d-bf57-1342e816791e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq\" (UID: \"d8b3eda9-d907-444d-bf57-1342e816791e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"
Jul 24 18:47:49.795061 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.795032    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d8b3eda9-d907-444d-bf57-1342e816791e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq\" (UID: \"d8b3eda9-d907-444d-bf57-1342e816791e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"
Jul 24 18:47:49.815973 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.815944    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gntfw\" (UniqueName: \"kubernetes.io/projected/d8b3eda9-d907-444d-bf57-1342e816791e-kube-api-access-gntfw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq\" (UID: \"d8b3eda9-d907-444d-bf57-1342e816791e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"
Jul 24 18:47:49.965763 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:49.965673    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"
Jul 24 18:47:50.117213 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:50.117183    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"]
Jul 24 18:47:50.119999 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:47:50.119977    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8b3eda9_d907_444d_bf57_1342e816791e.slice/crio-5dd53107ef5443aa2402a4ee7e7868d328f31b0b3bfe70264fede7d3a71c9c29 WatchSource:0}: Error finding container 5dd53107ef5443aa2402a4ee7e7868d328f31b0b3bfe70264fede7d3a71c9c29: Status 404 returned error can't find the container with id 5dd53107ef5443aa2402a4ee7e7868d328f31b0b3bfe70264fede7d3a71c9c29
Jul 24 18:47:51.034992 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.034953    2563 generic.go:356] "Generic (PLEG): container finished" podID="d8b3eda9-d907-444d-bf57-1342e816791e" containerID="ba7d13224b1c01a2980b66b3ec52a7f4bb68a0da91d6bd2716149baa93bc9b79" exitCode=0
Jul 24 18:47:51.035160 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.035040    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq" event={"ID":"d8b3eda9-d907-444d-bf57-1342e816791e","Type":"ContainerDied","Data":"ba7d13224b1c01a2980b66b3ec52a7f4bb68a0da91d6bd2716149baa93bc9b79"}
Jul 24 18:47:51.035160 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.035095    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq" event={"ID":"d8b3eda9-d907-444d-bf57-1342e816791e","Type":"ContainerStarted","Data":"5dd53107ef5443aa2402a4ee7e7868d328f31b0b3bfe70264fede7d3a71c9c29"}
Jul 24 18:47:51.684956 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.684914    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kltv7"]
Jul 24 18:47:51.687061 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.687044    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kltv7"
Jul 24 18:47:51.692006 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.691986    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-7s5d8\""
Jul 24 18:47:51.692137 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.692056    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 24 18:47:51.692334 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.692315    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 24 18:47:51.713498 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.713471    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kltv7"]
Jul 24 18:47:51.808581 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.808540    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgs2z\" (UniqueName: \"kubernetes.io/projected/522bf1ae-2dff-491a-ada9-1c1cefe075f1-kube-api-access-lgs2z\") pod \"cert-manager-cainjector-559cdd8c87-kltv7\" (UID: \"522bf1ae-2dff-491a-ada9-1c1cefe075f1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kltv7"
Jul 24 18:47:51.808766 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.808605    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/522bf1ae-2dff-491a-ada9-1c1cefe075f1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kltv7\" (UID: \"522bf1ae-2dff-491a-ada9-1c1cefe075f1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kltv7"
Jul 24 18:47:51.833493 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.833453    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kw8tt"]
Jul 24 18:47:51.837493 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.837386    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kw8tt"
Jul 24 18:47:51.841911 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.841886    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-qkpkh\""
Jul 24 18:47:51.848137 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.848111    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kw8tt"]
Jul 24 18:47:51.909165 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.909133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/522bf1ae-2dff-491a-ada9-1c1cefe075f1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kltv7\" (UID: \"522bf1ae-2dff-491a-ada9-1c1cefe075f1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kltv7"
Jul 24 18:47:51.909350 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.909183    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65e9540b-3111-48ee-b9e0-6801dedc6cd0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kw8tt\" (UID: \"65e9540b-3111-48ee-b9e0-6801dedc6cd0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kw8tt"
Jul 24 18:47:51.909350 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.909248    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n9kl\" (UniqueName: \"kubernetes.io/projected/65e9540b-3111-48ee-b9e0-6801dedc6cd0-kube-api-access-9n9kl\") pod \"cert-manager-webhook-6486ffd997-kw8tt\" (UID: \"65e9540b-3111-48ee-b9e0-6801dedc6cd0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kw8tt"
Jul 24 18:47:51.909350 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.909276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgs2z\" (UniqueName: \"kubernetes.io/projected/522bf1ae-2dff-491a-ada9-1c1cefe075f1-kube-api-access-lgs2z\") pod \"cert-manager-cainjector-559cdd8c87-kltv7\" (UID: \"522bf1ae-2dff-491a-ada9-1c1cefe075f1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kltv7"
Jul 24 18:47:51.923863 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.923830    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/522bf1ae-2dff-491a-ada9-1c1cefe075f1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kltv7\" (UID: \"522bf1ae-2dff-491a-ada9-1c1cefe075f1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kltv7"
Jul 24 18:47:51.924015 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:51.923908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgs2z\" (UniqueName: \"kubernetes.io/projected/522bf1ae-2dff-491a-ada9-1c1cefe075f1-kube-api-access-lgs2z\") pod \"cert-manager-cainjector-559cdd8c87-kltv7\" (UID: \"522bf1ae-2dff-491a-ada9-1c1cefe075f1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kltv7"
Jul 24 18:47:52.010561 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:52.010465    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65e9540b-3111-48ee-b9e0-6801dedc6cd0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kw8tt\" (UID: \"65e9540b-3111-48ee-b9e0-6801dedc6cd0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kw8tt"
Jul 24 18:47:52.010561 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:52.010529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n9kl\" (UniqueName: \"kubernetes.io/projected/65e9540b-3111-48ee-b9e0-6801dedc6cd0-kube-api-access-9n9kl\") pod \"cert-manager-webhook-6486ffd997-kw8tt\" (UID: \"65e9540b-3111-48ee-b9e0-6801dedc6cd0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kw8tt"
Jul 24 18:47:52.015337 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:52.015311    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kltv7"
Jul 24 18:47:52.021005 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:52.020980    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n9kl\" (UniqueName: \"kubernetes.io/projected/65e9540b-3111-48ee-b9e0-6801dedc6cd0-kube-api-access-9n9kl\") pod \"cert-manager-webhook-6486ffd997-kw8tt\" (UID: \"65e9540b-3111-48ee-b9e0-6801dedc6cd0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kw8tt"
Jul 24 18:47:52.021187 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:52.021167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65e9540b-3111-48ee-b9e0-6801dedc6cd0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kw8tt\" (UID: \"65e9540b-3111-48ee-b9e0-6801dedc6cd0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kw8tt"
Jul 24 18:47:52.148944 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:52.148912    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kw8tt"
Jul 24 18:47:52.168130 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:52.168099    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kltv7"]
Jul 24 18:47:52.171139 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:47:52.171113    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod522bf1ae_2dff_491a_ada9_1c1cefe075f1.slice/crio-71b5ce6abe17f610726f9f2e552fc425e99cc8d0fc135964907ad6b3c369849c WatchSource:0}: Error finding container 71b5ce6abe17f610726f9f2e552fc425e99cc8d0fc135964907ad6b3c369849c: Status 404 returned error can't find the container with id 71b5ce6abe17f610726f9f2e552fc425e99cc8d0fc135964907ad6b3c369849c
Jul 24 18:47:52.302524 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:52.302431    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kw8tt"]
Jul 24 18:47:52.305775 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:47:52.305743    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65e9540b_3111_48ee_b9e0_6801dedc6cd0.slice/crio-ae02bafd4960c5fcd2af294f8b92a21556a2a0f7327a2ef61c05b5cdadd77328 WatchSource:0}: Error finding container ae02bafd4960c5fcd2af294f8b92a21556a2a0f7327a2ef61c05b5cdadd77328: Status 404 returned error can't find the container with id ae02bafd4960c5fcd2af294f8b92a21556a2a0f7327a2ef61c05b5cdadd77328
Jul 24 18:47:53.045562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:53.045522    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kltv7" event={"ID":"522bf1ae-2dff-491a-ada9-1c1cefe075f1","Type":"ContainerStarted","Data":"71b5ce6abe17f610726f9f2e552fc425e99cc8d0fc135964907ad6b3c369849c"}
Jul 24 18:47:53.046792 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:53.046763    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kw8tt" event={"ID":"65e9540b-3111-48ee-b9e0-6801dedc6cd0","Type":"ContainerStarted","Data":"ae02bafd4960c5fcd2af294f8b92a21556a2a0f7327a2ef61c05b5cdadd77328"}
Jul 24 18:47:54.054106 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:54.054066    2563 generic.go:356] "Generic (PLEG): container finished" podID="d8b3eda9-d907-444d-bf57-1342e816791e" containerID="21275408a74696b9bdbbe0fea7ebea91157e71a9c80bd802b344ec0434fafd68" exitCode=0
Jul 24 18:47:54.054589 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:54.054176    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq" event={"ID":"d8b3eda9-d907-444d-bf57-1342e816791e","Type":"ContainerDied","Data":"21275408a74696b9bdbbe0fea7ebea91157e71a9c80bd802b344ec0434fafd68"}
Jul 24 18:47:55.061483 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:55.061441    2563 generic.go:356] "Generic (PLEG): container finished" podID="d8b3eda9-d907-444d-bf57-1342e816791e" containerID="4fb7c6268cdde1f3ea5f917d6ce0264c4fe946a65be2a4c4997a6cfb35083086" exitCode=0
Jul 24 18:47:55.061939 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:55.061541    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq" event={"ID":"d8b3eda9-d907-444d-bf57-1342e816791e","Type":"ContainerDied","Data":"4fb7c6268cdde1f3ea5f917d6ce0264c4fe946a65be2a4c4997a6cfb35083086"}
Jul 24 18:47:56.067162 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:56.067127    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kltv7" event={"ID":"522bf1ae-2dff-491a-ada9-1c1cefe075f1","Type":"ContainerStarted","Data":"acf242df55728fd66cfea241300b4d1b9f7a535a80df6d59d7cb43d4a28e4ae7"}
Jul 24 18:47:56.068694 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:56.068658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kw8tt" event={"ID":"65e9540b-3111-48ee-b9e0-6801dedc6cd0","Type":"ContainerStarted","Data":"c5f5225116757b8764f4e2b098a0565e7d60b0734b6f68a581f2f2757be2189a"}
Jul 24 18:47:56.083351 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:56.083301    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kltv7" podStartSLOduration=2.055835754 podStartE2EDuration="5.083286392s" totalImagesPullingTime="3.027450638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:47:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:47:56.082081242 +0000 UTC m=+214.473032718" watchObservedRunningTime="2026-07-24 18:47:56.083286392 +0000 UTC m=+214.474237868"
Jul 24 18:47:56.100414 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:56.100333    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-kw8tt" podStartSLOduration=2.19878369 podStartE2EDuration="5.10030886s" totalImagesPullingTime="2.90152517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:47:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:47:56.099633813 +0000 UTC m=+214.490585292" watchObservedRunningTime="2026-07-24 18:47:56.10030886 +0000 UTC m=+214.491260338"
Jul 24 18:47:56.191926 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:56.191896    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"
Jul 24 18:47:56.248875 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:56.248844    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d8b3eda9-d907-444d-bf57-1342e816791e-bundle\") pod \"d8b3eda9-d907-444d-bf57-1342e816791e\" (UID: \"d8b3eda9-d907-444d-bf57-1342e816791e\") "
Jul 24 18:47:56.249034 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:56.248940    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gntfw\" (UniqueName: \"kubernetes.io/projected/d8b3eda9-d907-444d-bf57-1342e816791e-kube-api-access-gntfw\") pod \"d8b3eda9-d907-444d-bf57-1342e816791e\" (UID: \"d8b3eda9-d907-444d-bf57-1342e816791e\") "
Jul 24 18:47:56.249034 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:56.248966    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d8b3eda9-d907-444d-bf57-1342e816791e-util\") pod \"d8b3eda9-d907-444d-bf57-1342e816791e\" (UID: \"d8b3eda9-d907-444d-bf57-1342e816791e\") "
Jul 24 18:47:56.249310 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:56.249285    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8b3eda9-d907-444d-bf57-1342e816791e-bundle" (OuterVolumeSpecName: "bundle") pod "d8b3eda9-d907-444d-bf57-1342e816791e" (UID: "d8b3eda9-d907-444d-bf57-1342e816791e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:47:56.251170 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:56.251138    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8b3eda9-d907-444d-bf57-1342e816791e-kube-api-access-gntfw" (OuterVolumeSpecName: "kube-api-access-gntfw") pod "d8b3eda9-d907-444d-bf57-1342e816791e" (UID: "d8b3eda9-d907-444d-bf57-1342e816791e"). InnerVolumeSpecName "kube-api-access-gntfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:47:56.253502 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:56.253467    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8b3eda9-d907-444d-bf57-1342e816791e-util" (OuterVolumeSpecName: "util") pod "d8b3eda9-d907-444d-bf57-1342e816791e" (UID: "d8b3eda9-d907-444d-bf57-1342e816791e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:47:56.350475 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:56.350367    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gntfw\" (UniqueName: \"kubernetes.io/projected/d8b3eda9-d907-444d-bf57-1342e816791e-kube-api-access-gntfw\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:47:56.350475 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:56.350408    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d8b3eda9-d907-444d-bf57-1342e816791e-util\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:47:56.350475 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:56.350417    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d8b3eda9-d907-444d-bf57-1342e816791e-bundle\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:47:57.073901 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:57.073867    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq"
Jul 24 18:47:57.073901 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:57.073892    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2kmq" event={"ID":"d8b3eda9-d907-444d-bf57-1342e816791e","Type":"ContainerDied","Data":"5dd53107ef5443aa2402a4ee7e7868d328f31b0b3bfe70264fede7d3a71c9c29"}
Jul 24 18:47:57.074344 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:57.073969    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5dd53107ef5443aa2402a4ee7e7868d328f31b0b3bfe70264fede7d3a71c9c29"
Jul 24 18:47:57.074344 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:57.074260    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kw8tt"
Jul 24 18:47:59.863940 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:59.863899    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wvtqt"]
Jul 24 18:47:59.864299 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:59.864148    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8b3eda9-d907-444d-bf57-1342e816791e" containerName="extract"
Jul 24 18:47:59.864299 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:59.864161    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8b3eda9-d907-444d-bf57-1342e816791e" containerName="extract"
Jul 24 18:47:59.864299 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:59.864173    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8b3eda9-d907-444d-bf57-1342e816791e" containerName="util"
Jul 24 18:47:59.864299 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:59.864178    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8b3eda9-d907-444d-bf57-1342e816791e" containerName="util"
Jul 24 18:47:59.864299 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:59.864186    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8b3eda9-d907-444d-bf57-1342e816791e" containerName="pull"
Jul 24 18:47:59.864299 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:59.864191    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8b3eda9-d907-444d-bf57-1342e816791e" containerName="pull"
Jul 24 18:47:59.864299 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:59.864237    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8b3eda9-d907-444d-bf57-1342e816791e" containerName="extract"
Jul 24 18:47:59.866425 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:59.866407    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wvtqt"
Jul 24 18:47:59.868859 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:59.868839    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-c2vv5\""
Jul 24 18:47:59.876465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:59.876445    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wvtqt"]
Jul 24 18:47:59.979102 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:59.979066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be7d89ce-d918-4cac-89cc-32c25a5dfef9-bound-sa-token\") pod \"cert-manager-55b4d954d5-wvtqt\" (UID: \"be7d89ce-d918-4cac-89cc-32c25a5dfef9\") " pod="cert-manager/cert-manager-55b4d954d5-wvtqt"
Jul 24 18:47:59.979271 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:47:59.979148    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85hbg\" (UniqueName: \"kubernetes.io/projected/be7d89ce-d918-4cac-89cc-32c25a5dfef9-kube-api-access-85hbg\") pod \"cert-manager-55b4d954d5-wvtqt\" (UID: \"be7d89ce-d918-4cac-89cc-32c25a5dfef9\") " pod="cert-manager/cert-manager-55b4d954d5-wvtqt"
Jul 24 18:48:00.079582 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:00.079540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85hbg\" (UniqueName: \"kubernetes.io/projected/be7d89ce-d918-4cac-89cc-32c25a5dfef9-kube-api-access-85hbg\") pod \"cert-manager-55b4d954d5-wvtqt\" (UID: \"be7d89ce-d918-4cac-89cc-32c25a5dfef9\") " pod="cert-manager/cert-manager-55b4d954d5-wvtqt"
Jul 24 18:48:00.079582 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:00.079586    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be7d89ce-d918-4cac-89cc-32c25a5dfef9-bound-sa-token\") pod \"cert-manager-55b4d954d5-wvtqt\" (UID: \"be7d89ce-d918-4cac-89cc-32c25a5dfef9\") " pod="cert-manager/cert-manager-55b4d954d5-wvtqt"
Jul 24 18:48:00.088882 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:00.088850    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85hbg\" (UniqueName: \"kubernetes.io/projected/be7d89ce-d918-4cac-89cc-32c25a5dfef9-kube-api-access-85hbg\") pod \"cert-manager-55b4d954d5-wvtqt\" (UID: \"be7d89ce-d918-4cac-89cc-32c25a5dfef9\") " pod="cert-manager/cert-manager-55b4d954d5-wvtqt"
Jul 24 18:48:00.089654 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:00.089636    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be7d89ce-d918-4cac-89cc-32c25a5dfef9-bound-sa-token\") pod \"cert-manager-55b4d954d5-wvtqt\" (UID: \"be7d89ce-d918-4cac-89cc-32c25a5dfef9\") " pod="cert-manager/cert-manager-55b4d954d5-wvtqt"
Jul 24 18:48:00.174802 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:00.174719    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wvtqt"
Jul 24 18:48:00.303118 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:00.303043    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wvtqt"]
Jul 24 18:48:00.305509 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:48:00.305483    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe7d89ce_d918_4cac_89cc_32c25a5dfef9.slice/crio-3cc4063f977daf9bb4b5d1bac27dc5d432302ce2ac6ea010a0fb98db27ad0629 WatchSource:0}: Error finding container 3cc4063f977daf9bb4b5d1bac27dc5d432302ce2ac6ea010a0fb98db27ad0629: Status 404 returned error can't find the container with id 3cc4063f977daf9bb4b5d1bac27dc5d432302ce2ac6ea010a0fb98db27ad0629
Jul 24 18:48:01.090886 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:01.090844    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wvtqt" event={"ID":"be7d89ce-d918-4cac-89cc-32c25a5dfef9","Type":"ContainerStarted","Data":"f07e43de6ed4b5d6d98509ef8744cd176855410bdc103ae8afd747e6349115d2"}
Jul 24 18:48:01.091261 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:01.090893    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wvtqt" event={"ID":"be7d89ce-d918-4cac-89cc-32c25a5dfef9","Type":"ContainerStarted","Data":"3cc4063f977daf9bb4b5d1bac27dc5d432302ce2ac6ea010a0fb98db27ad0629"}
Jul 24 18:48:01.107817 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:01.107768    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-wvtqt" podStartSLOduration=2.107751966 podStartE2EDuration="2.107751966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:47:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:48:01.107352924 +0000 UTC m=+219.498304405" watchObservedRunningTime="2026-07-24 18:48:01.107751966 +0000 UTC m=+219.498703443"
Jul 24 18:48:03.079697 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:03.079667    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kw8tt"
Jul 24 18:48:07.453058 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:07.453021    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dgwwl"]
Jul 24 18:48:07.455297 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:07.455281    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dgwwl"
Jul 24 18:48:07.458154 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:07.458127    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-rmhtx\""
Jul 24 18:48:07.458386 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:07.458371    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 18:48:07.459015 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:07.459001    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 18:48:07.467195 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:07.467169    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dgwwl"]
Jul 24 18:48:07.536168 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:07.536134    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c63da4d-799c-41e9-bfb0-44a78d4dfbc8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-dgwwl\" (UID: \"0c63da4d-799c-41e9-bfb0-44a78d4dfbc8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dgwwl"
Jul 24 18:48:07.536358 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:07.536202    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89zbk\" (UniqueName: \"kubernetes.io/projected/0c63da4d-799c-41e9-bfb0-44a78d4dfbc8-kube-api-access-89zbk\") pod \"openshift-lws-operator-fd8ccff4c-dgwwl\" (UID: \"0c63da4d-799c-41e9-bfb0-44a78d4dfbc8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dgwwl"
Jul 24 18:48:07.637539 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:07.637502    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89zbk\" (UniqueName: \"kubernetes.io/projected/0c63da4d-799c-41e9-bfb0-44a78d4dfbc8-kube-api-access-89zbk\") pod \"openshift-lws-operator-fd8ccff4c-dgwwl\" (UID: \"0c63da4d-799c-41e9-bfb0-44a78d4dfbc8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dgwwl"
Jul 24 18:48:07.637699 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:07.637548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c63da4d-799c-41e9-bfb0-44a78d4dfbc8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-dgwwl\" (UID: \"0c63da4d-799c-41e9-bfb0-44a78d4dfbc8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dgwwl"
Jul 24 18:48:07.637860 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:07.637844    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c63da4d-799c-41e9-bfb0-44a78d4dfbc8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-dgwwl\" (UID: \"0c63da4d-799c-41e9-bfb0-44a78d4dfbc8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dgwwl"
Jul 24 18:48:07.646152 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:07.646122    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89zbk\" (UniqueName: \"kubernetes.io/projected/0c63da4d-799c-41e9-bfb0-44a78d4dfbc8-kube-api-access-89zbk\") pod \"openshift-lws-operator-fd8ccff4c-dgwwl\" (UID: \"0c63da4d-799c-41e9-bfb0-44a78d4dfbc8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dgwwl"
Jul 24 18:48:07.764960 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:07.764858    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dgwwl"
Jul 24 18:48:07.899179 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:07.899141    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dgwwl"]
Jul 24 18:48:07.902477 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:48:07.902442    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c63da4d_799c_41e9_bfb0_44a78d4dfbc8.slice/crio-30a1de37516ff22d65336635317843fb998be33564271f369fcbbf1369cb96bd WatchSource:0}: Error finding container 30a1de37516ff22d65336635317843fb998be33564271f369fcbbf1369cb96bd: Status 404 returned error can't find the container with id 30a1de37516ff22d65336635317843fb998be33564271f369fcbbf1369cb96bd
Jul 24 18:48:08.120097 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:08.120062    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dgwwl" event={"ID":"0c63da4d-799c-41e9-bfb0-44a78d4dfbc8","Type":"ContainerStarted","Data":"30a1de37516ff22d65336635317843fb998be33564271f369fcbbf1369cb96bd"}
Jul 24 18:48:10.130594 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:10.130547    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dgwwl" event={"ID":"0c63da4d-799c-41e9-bfb0-44a78d4dfbc8","Type":"ContainerStarted","Data":"f4bf2e9ac0291bc187de42f0db18d9e8442734fb8214648b37f3b100206b0c3a"}
Jul 24 18:48:10.148960 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:10.148847    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dgwwl" podStartSLOduration=1.168199711 podStartE2EDuration="3.148826078s" totalImagesPullingTime="1.980626367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:48:10.1468395 +0000 UTC m=+228.537790969" watchObservedRunningTime="2026-07-24 18:48:10.148826078 +0000 UTC m=+228.539777558"
Jul 24 18:48:13.131456 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.131424    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"]
Jul 24 18:48:13.133606 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.133589    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"
Jul 24 18:48:13.135980 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.135952    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 18:48:13.136118 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.135983    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mkm8h\""
Jul 24 18:48:13.136118 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.136012    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 18:48:13.143850 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.143819    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"]
Jul 24 18:48:13.281990 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.281953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwxqt\" (UniqueName: \"kubernetes.io/projected/06ed44be-bd17-4dc7-bcd3-780034582bb2-kube-api-access-dwxqt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw\" (UID: \"06ed44be-bd17-4dc7-bcd3-780034582bb2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"
Jul 24 18:48:13.282155 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.282008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06ed44be-bd17-4dc7-bcd3-780034582bb2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw\" (UID: \"06ed44be-bd17-4dc7-bcd3-780034582bb2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"
Jul 24 18:48:13.282155 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.282068    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06ed44be-bd17-4dc7-bcd3-780034582bb2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw\" (UID: \"06ed44be-bd17-4dc7-bcd3-780034582bb2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"
Jul 24 18:48:13.382906 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.382789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwxqt\" (UniqueName: \"kubernetes.io/projected/06ed44be-bd17-4dc7-bcd3-780034582bb2-kube-api-access-dwxqt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw\" (UID: \"06ed44be-bd17-4dc7-bcd3-780034582bb2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"
Jul 24 18:48:13.382906 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.382886    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06ed44be-bd17-4dc7-bcd3-780034582bb2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw\" (UID: \"06ed44be-bd17-4dc7-bcd3-780034582bb2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"
Jul 24 18:48:13.382906 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.382920    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06ed44be-bd17-4dc7-bcd3-780034582bb2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw\" (UID: \"06ed44be-bd17-4dc7-bcd3-780034582bb2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"
Jul 24 18:48:13.383350 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.383329    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06ed44be-bd17-4dc7-bcd3-780034582bb2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw\" (UID: \"06ed44be-bd17-4dc7-bcd3-780034582bb2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"
Jul 24 18:48:13.383430 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.383344    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06ed44be-bd17-4dc7-bcd3-780034582bb2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw\" (UID: \"06ed44be-bd17-4dc7-bcd3-780034582bb2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"
Jul 24 18:48:13.391446 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.391413    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwxqt\" (UniqueName: \"kubernetes.io/projected/06ed44be-bd17-4dc7-bcd3-780034582bb2-kube-api-access-dwxqt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw\" (UID: \"06ed44be-bd17-4dc7-bcd3-780034582bb2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"
Jul 24 18:48:13.443381 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.443339    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"
Jul 24 18:48:13.571596 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:13.571553    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"]
Jul 24 18:48:13.574199 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:48:13.574169    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06ed44be_bd17_4dc7_bcd3_780034582bb2.slice/crio-2317cc53f8ec949bf37b2714c77081b5ecf54e68d6024048371c9ceeaaa55620 WatchSource:0}: Error finding container 2317cc53f8ec949bf37b2714c77081b5ecf54e68d6024048371c9ceeaaa55620: Status 404 returned error can't find the container with id 2317cc53f8ec949bf37b2714c77081b5ecf54e68d6024048371c9ceeaaa55620
Jul 24 18:48:14.147176 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:14.147139    2563 generic.go:356] "Generic (PLEG): container finished" podID="06ed44be-bd17-4dc7-bcd3-780034582bb2" containerID="eb236a1e7eedb72de77b4ebffad0fafa54baa944d4a541930db8b028e7f3d3f8" exitCode=0
Jul 24 18:48:14.147596 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:14.147224    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw" event={"ID":"06ed44be-bd17-4dc7-bcd3-780034582bb2","Type":"ContainerDied","Data":"eb236a1e7eedb72de77b4ebffad0fafa54baa944d4a541930db8b028e7f3d3f8"}
Jul 24 18:48:14.147596 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:14.147273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw" event={"ID":"06ed44be-bd17-4dc7-bcd3-780034582bb2","Type":"ContainerStarted","Data":"2317cc53f8ec949bf37b2714c77081b5ecf54e68d6024048371c9ceeaaa55620"}
Jul 24 18:48:15.153004 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:15.152967    2563 generic.go:356] "Generic (PLEG): container finished" podID="06ed44be-bd17-4dc7-bcd3-780034582bb2" containerID="0d71b9289c5272b3a1f9860034dc6bc7c249a834e6a14be3c1ec5e44593b7a98" exitCode=0
Jul 24 18:48:15.153415 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:15.153038    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw" event={"ID":"06ed44be-bd17-4dc7-bcd3-780034582bb2","Type":"ContainerDied","Data":"0d71b9289c5272b3a1f9860034dc6bc7c249a834e6a14be3c1ec5e44593b7a98"}
Jul 24 18:48:16.159472 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:16.159437    2563 generic.go:356] "Generic (PLEG): container finished" podID="06ed44be-bd17-4dc7-bcd3-780034582bb2" containerID="efacc9430a5e414285f9799af24573a5dce245cb4f40b51d8acfe3d38655fd3a" exitCode=0
Jul 24 18:48:16.159867 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:16.159506    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw" event={"ID":"06ed44be-bd17-4dc7-bcd3-780034582bb2","Type":"ContainerDied","Data":"efacc9430a5e414285f9799af24573a5dce245cb4f40b51d8acfe3d38655fd3a"}
Jul 24 18:48:17.296915 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:17.296882    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"
Jul 24 18:48:17.414318 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:17.414283    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06ed44be-bd17-4dc7-bcd3-780034582bb2-bundle\") pod \"06ed44be-bd17-4dc7-bcd3-780034582bb2\" (UID: \"06ed44be-bd17-4dc7-bcd3-780034582bb2\") "
Jul 24 18:48:17.414539 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:17.414363    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwxqt\" (UniqueName: \"kubernetes.io/projected/06ed44be-bd17-4dc7-bcd3-780034582bb2-kube-api-access-dwxqt\") pod \"06ed44be-bd17-4dc7-bcd3-780034582bb2\" (UID: \"06ed44be-bd17-4dc7-bcd3-780034582bb2\") "
Jul 24 18:48:17.414539 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:17.414413    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06ed44be-bd17-4dc7-bcd3-780034582bb2-util\") pod \"06ed44be-bd17-4dc7-bcd3-780034582bb2\" (UID: \"06ed44be-bd17-4dc7-bcd3-780034582bb2\") "
Jul 24 18:48:17.414981 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:17.414954    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06ed44be-bd17-4dc7-bcd3-780034582bb2-bundle" (OuterVolumeSpecName: "bundle") pod "06ed44be-bd17-4dc7-bcd3-780034582bb2" (UID: "06ed44be-bd17-4dc7-bcd3-780034582bb2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:48:17.416590 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:17.416556    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06ed44be-bd17-4dc7-bcd3-780034582bb2-kube-api-access-dwxqt" (OuterVolumeSpecName: "kube-api-access-dwxqt") pod "06ed44be-bd17-4dc7-bcd3-780034582bb2" (UID: "06ed44be-bd17-4dc7-bcd3-780034582bb2"). InnerVolumeSpecName "kube-api-access-dwxqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:48:17.419837 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:17.419785    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06ed44be-bd17-4dc7-bcd3-780034582bb2-util" (OuterVolumeSpecName: "util") pod "06ed44be-bd17-4dc7-bcd3-780034582bb2" (UID: "06ed44be-bd17-4dc7-bcd3-780034582bb2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:48:17.515473 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:17.515435    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwxqt\" (UniqueName: \"kubernetes.io/projected/06ed44be-bd17-4dc7-bcd3-780034582bb2-kube-api-access-dwxqt\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:48:17.515473 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:17.515466    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06ed44be-bd17-4dc7-bcd3-780034582bb2-util\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:48:17.515473 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:17.515475    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06ed44be-bd17-4dc7-bcd3-780034582bb2-bundle\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:48:18.175743 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:18.175715    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw"
Jul 24 18:48:18.175743 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:18.175724    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af58mlw" event={"ID":"06ed44be-bd17-4dc7-bcd3-780034582bb2","Type":"ContainerDied","Data":"2317cc53f8ec949bf37b2714c77081b5ecf54e68d6024048371c9ceeaaa55620"}
Jul 24 18:48:18.175989 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:18.175765    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2317cc53f8ec949bf37b2714c77081b5ecf54e68d6024048371c9ceeaaa55620"
Jul 24 18:48:27.201761 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.201729    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-594d74945-klg86"]
Jul 24 18:48:27.202204 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.201957    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06ed44be-bd17-4dc7-bcd3-780034582bb2" containerName="pull"
Jul 24 18:48:27.202204 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.201966    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="06ed44be-bd17-4dc7-bcd3-780034582bb2" containerName="pull"
Jul 24 18:48:27.202204 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.201980    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06ed44be-bd17-4dc7-bcd3-780034582bb2" containerName="extract"
Jul 24 18:48:27.202204 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.201985    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="06ed44be-bd17-4dc7-bcd3-780034582bb2" containerName="extract"
Jul 24 18:48:27.202204 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.201994    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06ed44be-bd17-4dc7-bcd3-780034582bb2" containerName="util"
Jul 24 18:48:27.202204 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.201999    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="06ed44be-bd17-4dc7-bcd3-780034582bb2" containerName="util"
Jul 24 18:48:27.202204 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.202044    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="06ed44be-bd17-4dc7-bcd3-780034582bb2" containerName="extract"
Jul 24 18:48:27.255183 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.255139    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-594d74945-klg86"]
Jul 24 18:48:27.255183 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.255171    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:27.260897 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.260874    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4gzxp\""
Jul 24 18:48:27.267151 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.267124    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 18:48:27.267293 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.267275    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 18:48:27.267713 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.267695    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 18:48:27.392455 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.392410    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cc2476e-7fea-4b2b-ab47-9040834dba55-cert\") pod \"lws-controller-manager-594d74945-klg86\" (UID: \"6cc2476e-7fea-4b2b-ab47-9040834dba55\") " pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:27.392636 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.392466    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6cc2476e-7fea-4b2b-ab47-9040834dba55-manager-config\") pod \"lws-controller-manager-594d74945-klg86\" (UID: \"6cc2476e-7fea-4b2b-ab47-9040834dba55\") " pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:27.392636 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.392508    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6cc2476e-7fea-4b2b-ab47-9040834dba55-metrics-cert\") pod \"lws-controller-manager-594d74945-klg86\" (UID: \"6cc2476e-7fea-4b2b-ab47-9040834dba55\") " pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:27.392636 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.392526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcclv\" (UniqueName: \"kubernetes.io/projected/6cc2476e-7fea-4b2b-ab47-9040834dba55-kube-api-access-dcclv\") pod \"lws-controller-manager-594d74945-klg86\" (UID: \"6cc2476e-7fea-4b2b-ab47-9040834dba55\") " pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:27.493831 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.493745    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6cc2476e-7fea-4b2b-ab47-9040834dba55-metrics-cert\") pod \"lws-controller-manager-594d74945-klg86\" (UID: \"6cc2476e-7fea-4b2b-ab47-9040834dba55\") " pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:27.493831 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.493784    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcclv\" (UniqueName: \"kubernetes.io/projected/6cc2476e-7fea-4b2b-ab47-9040834dba55-kube-api-access-dcclv\") pod \"lws-controller-manager-594d74945-klg86\" (UID: \"6cc2476e-7fea-4b2b-ab47-9040834dba55\") " pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:27.494026 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.493979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cc2476e-7fea-4b2b-ab47-9040834dba55-cert\") pod \"lws-controller-manager-594d74945-klg86\" (UID: \"6cc2476e-7fea-4b2b-ab47-9040834dba55\") " pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:27.494083 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.494045    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6cc2476e-7fea-4b2b-ab47-9040834dba55-manager-config\") pod \"lws-controller-manager-594d74945-klg86\" (UID: \"6cc2476e-7fea-4b2b-ab47-9040834dba55\") " pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:27.494643 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.494620    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6cc2476e-7fea-4b2b-ab47-9040834dba55-manager-config\") pod \"lws-controller-manager-594d74945-klg86\" (UID: \"6cc2476e-7fea-4b2b-ab47-9040834dba55\") " pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:27.496382 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.496361    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cc2476e-7fea-4b2b-ab47-9040834dba55-cert\") pod \"lws-controller-manager-594d74945-klg86\" (UID: \"6cc2476e-7fea-4b2b-ab47-9040834dba55\") " pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:27.496488 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.496386    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6cc2476e-7fea-4b2b-ab47-9040834dba55-metrics-cert\") pod \"lws-controller-manager-594d74945-klg86\" (UID: \"6cc2476e-7fea-4b2b-ab47-9040834dba55\") " pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:27.509616 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.509588    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcclv\" (UniqueName: \"kubernetes.io/projected/6cc2476e-7fea-4b2b-ab47-9040834dba55-kube-api-access-dcclv\") pod \"lws-controller-manager-594d74945-klg86\" (UID: \"6cc2476e-7fea-4b2b-ab47-9040834dba55\") " pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:27.564638 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.564601    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:27.755442 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:27.755363    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-594d74945-klg86"]
Jul 24 18:48:27.758121 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:48:27.758077    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc2476e_7fea_4b2b_ab47_9040834dba55.slice/crio-9651dd6db182f93a301dadfa0190d9712679468aca676924cabebd8b7e4a4af9 WatchSource:0}: Error finding container 9651dd6db182f93a301dadfa0190d9712679468aca676924cabebd8b7e4a4af9: Status 404 returned error can't find the container with id 9651dd6db182f93a301dadfa0190d9712679468aca676924cabebd8b7e4a4af9
Jul 24 18:48:28.215452 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:28.215416    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86" event={"ID":"6cc2476e-7fea-4b2b-ab47-9040834dba55","Type":"ContainerStarted","Data":"9651dd6db182f93a301dadfa0190d9712679468aca676924cabebd8b7e4a4af9"}
Jul 24 18:48:29.221615 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.221575    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86" event={"ID":"6cc2476e-7fea-4b2b-ab47-9040834dba55","Type":"ContainerStarted","Data":"3680dec5d221d8ac022eaa42f4bdfaff192ad7bf53335a2a04a98fb7bec47229"}
Jul 24 18:48:29.221989 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.221722    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:29.246548 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.246489    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86" podStartSLOduration=0.928266896 podStartE2EDuration="2.246472332s" totalImagesPullingTime="1.318205436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:48:29.244891362 +0000 UTC m=+247.635842838" watchObservedRunningTime="2026-07-24 18:48:29.246472332 +0000 UTC m=+247.637423805"
Jul 24 18:48:29.707962 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.707929    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"]
Jul 24 18:48:29.710222 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.710206    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.714568 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.714543    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 18:48:29.714568 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.714559    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 18:48:29.714568 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.714544    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 18:48:29.714843 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.714829    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 18:48:29.717041 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.716949    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7rgj8\""
Jul 24 18:48:29.736616 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.736583    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"]
Jul 24 18:48:29.812339 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.812298    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e976c7fd-067a-48d2-a06e-f27bce7c1aba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.812565 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.812373    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e976c7fd-067a-48d2-a06e-f27bce7c1aba-manifests\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.812565 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.812459    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llj9v\" (UniqueName: \"kubernetes.io/projected/e976c7fd-067a-48d2-a06e-f27bce7c1aba-kube-api-access-llj9v\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.812565 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.812501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e976c7fd-067a-48d2-a06e-f27bce7c1aba-webhook-cert\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.812565 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.812533    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e976c7fd-067a-48d2-a06e-f27bce7c1aba-tmp\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.913288 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.913246    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e976c7fd-067a-48d2-a06e-f27bce7c1aba-manifests\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.913288 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.913293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llj9v\" (UniqueName: \"kubernetes.io/projected/e976c7fd-067a-48d2-a06e-f27bce7c1aba-kube-api-access-llj9v\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.913587 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.913318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e976c7fd-067a-48d2-a06e-f27bce7c1aba-webhook-cert\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.913587 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.913443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e976c7fd-067a-48d2-a06e-f27bce7c1aba-tmp\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.913587 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.913492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e976c7fd-067a-48d2-a06e-f27bce7c1aba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.913773 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.913753    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e976c7fd-067a-48d2-a06e-f27bce7c1aba-manifests\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.915847 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.915826    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e976c7fd-067a-48d2-a06e-f27bce7c1aba-tmp\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.915964 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.915945    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e976c7fd-067a-48d2-a06e-f27bce7c1aba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.916154 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.916132    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e976c7fd-067a-48d2-a06e-f27bce7c1aba-webhook-cert\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:29.927342 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:29.927312    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llj9v\" (UniqueName: \"kubernetes.io/projected/e976c7fd-067a-48d2-a06e-f27bce7c1aba-kube-api-access-llj9v\") pod \"opendatahub-operator-controller-manager-6559d49d8b-nkx69\" (UID: \"e976c7fd-067a-48d2-a06e-f27bce7c1aba\") " pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:30.020751 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:30.020660    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:30.163542 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:30.163518    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"]
Jul 24 18:48:30.165655 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:48:30.165627    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode976c7fd_067a_48d2_a06e_f27bce7c1aba.slice/crio-c24fdd6e8ecaa7e89e3ff4c1c750f8d2b308b70d33d50f1babd07a6b389e09a4 WatchSource:0}: Error finding container c24fdd6e8ecaa7e89e3ff4c1c750f8d2b308b70d33d50f1babd07a6b389e09a4: Status 404 returned error can't find the container with id c24fdd6e8ecaa7e89e3ff4c1c750f8d2b308b70d33d50f1babd07a6b389e09a4
Jul 24 18:48:30.227343 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:30.227307    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69" event={"ID":"e976c7fd-067a-48d2-a06e-f27bce7c1aba","Type":"ContainerStarted","Data":"c24fdd6e8ecaa7e89e3ff4c1c750f8d2b308b70d33d50f1babd07a6b389e09a4"}
Jul 24 18:48:33.245257 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:33.245217    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69" event={"ID":"e976c7fd-067a-48d2-a06e-f27bce7c1aba","Type":"ContainerStarted","Data":"9d7b3e0eb04894e6ef3601387ff4a3bf05015fe5b49fa2caaf05f6fe54b07d6f"}
Jul 24 18:48:34.251178 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:34.251141    2563 generic.go:356] "Generic (PLEG): container finished" podID="e976c7fd-067a-48d2-a06e-f27bce7c1aba" containerID="9d7b3e0eb04894e6ef3601387ff4a3bf05015fe5b49fa2caaf05f6fe54b07d6f" exitCode=0
Jul 24 18:48:34.251578 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:34.251216    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69" event={"ID":"e976c7fd-067a-48d2-a06e-f27bce7c1aba","Type":"ContainerDied","Data":"9d7b3e0eb04894e6ef3601387ff4a3bf05015fe5b49fa2caaf05f6fe54b07d6f"}
Jul 24 18:48:35.258099 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:35.258061    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69" event={"ID":"e976c7fd-067a-48d2-a06e-f27bce7c1aba","Type":"ContainerStarted","Data":"1944651cd4c701ba6757dfcd03ce10cafabcd9008191e25d178b19fdfcd4129b"}
Jul 24 18:48:35.258550 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:35.258191    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:35.282126 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:35.282076    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69" podStartSLOduration=2.060909621 podStartE2EDuration="6.28205433s" totalImagesPullingTime="3.212359809s" totalInitContainerRuntime="1.0087849s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:48:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:48:35.280713546 +0000 UTC m=+253.671665024" watchObservedRunningTime="2026-07-24 18:48:35.28205433 +0000 UTC m=+253.673005807"
Jul 24 18:48:40.229517 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:40.229485    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-594d74945-klg86"
Jul 24 18:48:46.264814 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:46.264782    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6559d49d8b-nkx69"
Jul 24 18:48:56.390034 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.389999    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"]
Jul 24 18:48:56.394833 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.394814    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.397428 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.397379    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 24 18:48:56.397568 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.397432    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 18:48:56.397568 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.397451    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hfrkk\""
Jul 24 18:48:56.397704 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.397683    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 24 18:48:56.397756 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.397701    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 18:48:56.397756 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.397683    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 24 18:48:56.397756 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.397742    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 18:48:56.405587 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.405566    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"]
Jul 24 18:48:56.517999 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.517957    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40d75a61-c418-46e0-8944-f5b8a38d48a2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.518193 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.518012    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/40d75a61-c418-46e0-8944-f5b8a38d48a2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.518193 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.518037    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr68q\" (UniqueName: \"kubernetes.io/projected/40d75a61-c418-46e0-8944-f5b8a38d48a2-kube-api-access-gr68q\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.518193 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.518088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/40d75a61-c418-46e0-8944-f5b8a38d48a2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.518193 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.518135    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/40d75a61-c418-46e0-8944-f5b8a38d48a2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.518193 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.518160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40d75a61-c418-46e0-8944-f5b8a38d48a2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.518372 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.518196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/40d75a61-c418-46e0-8944-f5b8a38d48a2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.619359 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.619311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40d75a61-c418-46e0-8944-f5b8a38d48a2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.619552 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.619371    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/40d75a61-c418-46e0-8944-f5b8a38d48a2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.619552 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.619430    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr68q\" (UniqueName: \"kubernetes.io/projected/40d75a61-c418-46e0-8944-f5b8a38d48a2-kube-api-access-gr68q\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.619552 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.619458    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/40d75a61-c418-46e0-8944-f5b8a38d48a2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.619552 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.619502    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/40d75a61-c418-46e0-8944-f5b8a38d48a2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.619552 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.619532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40d75a61-c418-46e0-8944-f5b8a38d48a2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.619758 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.619569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/40d75a61-c418-46e0-8944-f5b8a38d48a2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.620267 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.620242    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/40d75a61-c418-46e0-8944-f5b8a38d48a2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.621949 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.621925    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/40d75a61-c418-46e0-8944-f5b8a38d48a2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.622091 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.622066    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40d75a61-c418-46e0-8944-f5b8a38d48a2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.622357 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.622330    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/40d75a61-c418-46e0-8944-f5b8a38d48a2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.622565 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.622547    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/40d75a61-c418-46e0-8944-f5b8a38d48a2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.631505 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.631482    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40d75a61-c418-46e0-8944-f5b8a38d48a2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.635570 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.635545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr68q\" (UniqueName: \"kubernetes.io/projected/40d75a61-c418-46e0-8944-f5b8a38d48a2-kube-api-access-gr68q\") pod \"istiod-openshift-gateway-5677ccb49b-q6pr5\" (UID: \"40d75a61-c418-46e0-8944-f5b8a38d48a2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.704446 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.704314    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:48:56.842933 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:56.842915    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"]
Jul 24 18:48:56.845425 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:48:56.845382    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40d75a61_c418_46e0_8944_f5b8a38d48a2.slice/crio-28a2d072af3e112cec4aae97dc6522297175b30569312a1231b280fb173cee70 WatchSource:0}: Error finding container 28a2d072af3e112cec4aae97dc6522297175b30569312a1231b280fb173cee70: Status 404 returned error can't find the container with id 28a2d072af3e112cec4aae97dc6522297175b30569312a1231b280fb173cee70
Jul 24 18:48:57.359711 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:57.359677    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5" event={"ID":"40d75a61-c418-46e0-8944-f5b8a38d48a2","Type":"ContainerStarted","Data":"28a2d072af3e112cec4aae97dc6522297175b30569312a1231b280fb173cee70"}
Jul 24 18:48:59.267670 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:59.267624    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 18:48:59.267923 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:59.267720    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 18:48:59.371910 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:59.371878    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5" event={"ID":"40d75a61-c418-46e0-8944-f5b8a38d48a2","Type":"ContainerStarted","Data":"a6cd3bea684dbb0be22577ae0e67c92107ed2f5bbc7eb8956b1d824dada2fcfe"}
Jul 24 18:48:59.372096 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:48:59.372002    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:49:00.378659 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.378610    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-q6pr5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 18:49:00.379027 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.378685    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5" podUID="40d75a61-c418-46e0-8944-f5b8a38d48a2" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 18:49:00.531614 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.531563    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5" podStartSLOduration=2.111535864 podStartE2EDuration="4.53154295s" totalImagesPullingTime="2.420007086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:48:59.45626173 +0000 UTC m=+277.847213219" watchObservedRunningTime="2026-07-24 18:49:00.53154295 +0000 UTC m=+278.922494439"
Jul 24 18:49:00.532640 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.532619    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l"]
Jul 24 18:49:00.537148 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.537124    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l"
Jul 24 18:49:00.547990 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.547968    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 24 18:49:00.548108 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.547968    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-lzwv6\""
Jul 24 18:49:00.594212 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.594177    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l"]
Jul 24 18:49:00.655049 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.654959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e360529-8929-437d-9d5b-c0c61d0c83a3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-78b6l\" (UID: \"8e360529-8929-437d-9d5b-c0c61d0c83a3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l"
Jul 24 18:49:00.655049 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.655022    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4bf7\" (UniqueName: \"kubernetes.io/projected/8e360529-8929-437d-9d5b-c0c61d0c83a3-kube-api-access-r4bf7\") pod \"llmisvc-controller-manager-777c6bd7d5-78b6l\" (UID: \"8e360529-8929-437d-9d5b-c0c61d0c83a3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l"
Jul 24 18:49:00.755865 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.755824    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e360529-8929-437d-9d5b-c0c61d0c83a3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-78b6l\" (UID: \"8e360529-8929-437d-9d5b-c0c61d0c83a3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l"
Jul 24 18:49:00.756077 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.755932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4bf7\" (UniqueName: \"kubernetes.io/projected/8e360529-8929-437d-9d5b-c0c61d0c83a3-kube-api-access-r4bf7\") pod \"llmisvc-controller-manager-777c6bd7d5-78b6l\" (UID: \"8e360529-8929-437d-9d5b-c0c61d0c83a3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l"
Jul 24 18:49:00.758894 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.758853    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e360529-8929-437d-9d5b-c0c61d0c83a3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-78b6l\" (UID: \"8e360529-8929-437d-9d5b-c0c61d0c83a3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l"
Jul 24 18:49:00.770761 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.770721    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4bf7\" (UniqueName: \"kubernetes.io/projected/8e360529-8929-437d-9d5b-c0c61d0c83a3-kube-api-access-r4bf7\") pod \"llmisvc-controller-manager-777c6bd7d5-78b6l\" (UID: \"8e360529-8929-437d-9d5b-c0c61d0c83a3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l"
Jul 24 18:49:00.846757 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:00.846723    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l"
Jul 24 18:49:01.007281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:01.007111    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l"]
Jul 24 18:49:01.009340 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:49:01.009315    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8e360529_8929_437d_9d5b_c0c61d0c83a3.slice/crio-e9ba6258d9d00dcefef0029a68182b4424cef5799d52ed9b72cd6edfa4f53b77 WatchSource:0}: Error finding container e9ba6258d9d00dcefef0029a68182b4424cef5799d52ed9b72cd6edfa4f53b77: Status 404 returned error can't find the container with id e9ba6258d9d00dcefef0029a68182b4424cef5799d52ed9b72cd6edfa4f53b77
Jul 24 18:49:01.382301 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:01.382264    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l" event={"ID":"8e360529-8929-437d-9d5b-c0c61d0c83a3","Type":"ContainerStarted","Data":"e9ba6258d9d00dcefef0029a68182b4424cef5799d52ed9b72cd6edfa4f53b77"}
Jul 24 18:49:03.378323 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:03.378276    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6pr5"
Jul 24 18:49:05.401226 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:05.401184    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l" event={"ID":"8e360529-8929-437d-9d5b-c0c61d0c83a3","Type":"ContainerStarted","Data":"0ad883d0e2315156de5a0bb86dbaaf6ccc853845c30a14de1337a1ccda7a393c"}
Jul 24 18:49:05.401732 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:05.401321    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l"
Jul 24 18:49:05.420443 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:05.420367    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l" podStartSLOduration=2.137011282 podStartE2EDuration="5.42035146s" totalImagesPullingTime="3.283340178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:49:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:49:05.418473333 +0000 UTC m=+283.809424811" watchObservedRunningTime="2026-07-24 18:49:05.42035146 +0000 UTC m=+283.811302936"
Jul 24 18:49:22.039892 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:22.039861    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 18:49:22.041783 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:22.041760    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 18:49:22.048471 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:22.048451    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 18:49:36.412641 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:49:36.412603    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-78b6l"
Jul 24 18:50:05.547281 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.547246    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"]
Jul 24 18:50:05.550857 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.550834    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"
Jul 24 18:50:05.553481 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.553462    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 18:50:05.554340 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.554306    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 18:50:05.554473 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.554341    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jwg7r\""
Jul 24 18:50:05.560234 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.560205    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"]
Jul 24 18:50:05.675693 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.675658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/511f5628-511a-4c8a-8d0b-88dc56223da3-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f\" (UID: \"511f5628-511a-4c8a-8d0b-88dc56223da3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"
Jul 24 18:50:05.675883 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.675715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmzzw\" (UniqueName: \"kubernetes.io/projected/511f5628-511a-4c8a-8d0b-88dc56223da3-kube-api-access-dmzzw\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f\" (UID: \"511f5628-511a-4c8a-8d0b-88dc56223da3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"
Jul 24 18:50:05.675883 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.675770    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/511f5628-511a-4c8a-8d0b-88dc56223da3-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f\" (UID: \"511f5628-511a-4c8a-8d0b-88dc56223da3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"
Jul 24 18:50:05.776789 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.776748    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/511f5628-511a-4c8a-8d0b-88dc56223da3-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f\" (UID: \"511f5628-511a-4c8a-8d0b-88dc56223da3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"
Jul 24 18:50:05.777001 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.776850    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmzzw\" (UniqueName: \"kubernetes.io/projected/511f5628-511a-4c8a-8d0b-88dc56223da3-kube-api-access-dmzzw\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f\" (UID: \"511f5628-511a-4c8a-8d0b-88dc56223da3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"
Jul 24 18:50:05.777001 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.776893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/511f5628-511a-4c8a-8d0b-88dc56223da3-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f\" (UID: \"511f5628-511a-4c8a-8d0b-88dc56223da3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"
Jul 24 18:50:05.777129 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.777078    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/511f5628-511a-4c8a-8d0b-88dc56223da3-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f\" (UID: \"511f5628-511a-4c8a-8d0b-88dc56223da3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"
Jul 24 18:50:05.777223 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.777203    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/511f5628-511a-4c8a-8d0b-88dc56223da3-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f\" (UID: \"511f5628-511a-4c8a-8d0b-88dc56223da3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"
Jul 24 18:50:05.785690 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.785668    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmzzw\" (UniqueName: \"kubernetes.io/projected/511f5628-511a-4c8a-8d0b-88dc56223da3-kube-api-access-dmzzw\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f\" (UID: \"511f5628-511a-4c8a-8d0b-88dc56223da3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"
Jul 24 18:50:05.860331 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.860292    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"
Jul 24 18:50:05.994628 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:05.994598    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"]
Jul 24 18:50:05.998587 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:50:05.998559    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511f5628_511a_4c8a_8d0b_88dc56223da3.slice/crio-dd3541069973bdc742f799924efc790a2c6e3287f087e1bcee1938d12a395e20 WatchSource:0}: Error finding container dd3541069973bdc742f799924efc790a2c6e3287f087e1bcee1938d12a395e20: Status 404 returned error can't find the container with id dd3541069973bdc742f799924efc790a2c6e3287f087e1bcee1938d12a395e20
Jul 24 18:50:06.144781 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.144750    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"]
Jul 24 18:50:06.147889 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.147871    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"
Jul 24 18:50:06.155557 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.155529    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"]
Jul 24 18:50:06.280976 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.280946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62188e01-a45c-4f10-bfba-24ecbf9c1000-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv\" (UID: \"62188e01-a45c-4f10-bfba-24ecbf9c1000\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"
Jul 24 18:50:06.280976 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.280982    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgqv5\" (UniqueName: \"kubernetes.io/projected/62188e01-a45c-4f10-bfba-24ecbf9c1000-kube-api-access-mgqv5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv\" (UID: \"62188e01-a45c-4f10-bfba-24ecbf9c1000\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"
Jul 24 18:50:06.281177 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.281016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62188e01-a45c-4f10-bfba-24ecbf9c1000-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv\" (UID: \"62188e01-a45c-4f10-bfba-24ecbf9c1000\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"
Jul 24 18:50:06.382041 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.381958    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62188e01-a45c-4f10-bfba-24ecbf9c1000-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv\" (UID: \"62188e01-a45c-4f10-bfba-24ecbf9c1000\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"
Jul 24 18:50:06.382041 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.381994    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgqv5\" (UniqueName: \"kubernetes.io/projected/62188e01-a45c-4f10-bfba-24ecbf9c1000-kube-api-access-mgqv5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv\" (UID: \"62188e01-a45c-4f10-bfba-24ecbf9c1000\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"
Jul 24 18:50:06.382250 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.382122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62188e01-a45c-4f10-bfba-24ecbf9c1000-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv\" (UID: \"62188e01-a45c-4f10-bfba-24ecbf9c1000\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"
Jul 24 18:50:06.382345 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.382329    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62188e01-a45c-4f10-bfba-24ecbf9c1000-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv\" (UID: \"62188e01-a45c-4f10-bfba-24ecbf9c1000\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"
Jul 24 18:50:06.382446 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.382431    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62188e01-a45c-4f10-bfba-24ecbf9c1000-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv\" (UID: \"62188e01-a45c-4f10-bfba-24ecbf9c1000\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"
Jul 24 18:50:06.390196 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.390177    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgqv5\" (UniqueName: \"kubernetes.io/projected/62188e01-a45c-4f10-bfba-24ecbf9c1000-kube-api-access-mgqv5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv\" (UID: \"62188e01-a45c-4f10-bfba-24ecbf9c1000\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"
Jul 24 18:50:06.500031 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.499989    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"
Jul 24 18:50:06.550228 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.547709    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"]
Jul 24 18:50:06.555280 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.555191    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"
Jul 24 18:50:06.558495 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.558470    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"]
Jul 24 18:50:06.635668 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.635594    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"]
Jul 24 18:50:06.637730 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:50:06.637701    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62188e01_a45c_4f10_bfba_24ecbf9c1000.slice/crio-97c98bb0f50bee1c7b102f93fa6274e3cf5eec6a9366f600418e3010e04cf6d5 WatchSource:0}: Error finding container 97c98bb0f50bee1c7b102f93fa6274e3cf5eec6a9366f600418e3010e04cf6d5: Status 404 returned error can't find the container with id 97c98bb0f50bee1c7b102f93fa6274e3cf5eec6a9366f600418e3010e04cf6d5
Jul 24 18:50:06.669623 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.669586    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv" event={"ID":"62188e01-a45c-4f10-bfba-24ecbf9c1000","Type":"ContainerStarted","Data":"97c98bb0f50bee1c7b102f93fa6274e3cf5eec6a9366f600418e3010e04cf6d5"}
Jul 24 18:50:06.671232 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.671166    2563 generic.go:356] "Generic (PLEG): container finished" podID="511f5628-511a-4c8a-8d0b-88dc56223da3" containerID="9b4a81e44c6114e3b0686f390e3260c1cb1ebace1b17c5bb4e4982a593691c1e" exitCode=0
Jul 24 18:50:06.671421 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.671254    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f" event={"ID":"511f5628-511a-4c8a-8d0b-88dc56223da3","Type":"ContainerDied","Data":"9b4a81e44c6114e3b0686f390e3260c1cb1ebace1b17c5bb4e4982a593691c1e"}
Jul 24 18:50:06.671421 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.671306    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f" event={"ID":"511f5628-511a-4c8a-8d0b-88dc56223da3","Type":"ContainerStarted","Data":"dd3541069973bdc742f799924efc790a2c6e3287f087e1bcee1938d12a395e20"}
Jul 24 18:50:06.671831 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.671814    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 18:50:06.684774 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.684744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9797f2d8-5332-4ea3-8033-422021d8bc0c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs\" (UID: \"9797f2d8-5332-4ea3-8033-422021d8bc0c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"
Jul 24 18:50:06.684893 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.684803    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9797f2d8-5332-4ea3-8033-422021d8bc0c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs\" (UID: \"9797f2d8-5332-4ea3-8033-422021d8bc0c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"
Jul 24 18:50:06.684893 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.684870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spqq4\" (UniqueName: \"kubernetes.io/projected/9797f2d8-5332-4ea3-8033-422021d8bc0c-kube-api-access-spqq4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs\" (UID: \"9797f2d8-5332-4ea3-8033-422021d8bc0c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"
Jul 24 18:50:06.785521 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.785478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9797f2d8-5332-4ea3-8033-422021d8bc0c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs\" (UID: \"9797f2d8-5332-4ea3-8033-422021d8bc0c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"
Jul 24 18:50:06.785717 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.785555    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spqq4\" (UniqueName: \"kubernetes.io/projected/9797f2d8-5332-4ea3-8033-422021d8bc0c-kube-api-access-spqq4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs\" (UID: \"9797f2d8-5332-4ea3-8033-422021d8bc0c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"
Jul 24 18:50:06.785717 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.785626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9797f2d8-5332-4ea3-8033-422021d8bc0c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs\" (UID: \"9797f2d8-5332-4ea3-8033-422021d8bc0c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"
Jul 24 18:50:06.785955 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.785934    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9797f2d8-5332-4ea3-8033-422021d8bc0c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs\" (UID: \"9797f2d8-5332-4ea3-8033-422021d8bc0c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"
Jul 24 18:50:06.786020 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.785937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9797f2d8-5332-4ea3-8033-422021d8bc0c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs\" (UID: \"9797f2d8-5332-4ea3-8033-422021d8bc0c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"
Jul 24 18:50:06.794102 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.794072    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spqq4\" (UniqueName: \"kubernetes.io/projected/9797f2d8-5332-4ea3-8033-422021d8bc0c-kube-api-access-spqq4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs\" (UID: \"9797f2d8-5332-4ea3-8033-422021d8bc0c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"
Jul 24 18:50:06.870115 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.870071    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"
Jul 24 18:50:06.950608 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.950575    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"]
Jul 24 18:50:06.954546 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.954259    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"
Jul 24 18:50:06.963003 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.962976    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"]
Jul 24 18:50:06.987623 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.987592    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4qmn\" (UniqueName: \"kubernetes.io/projected/10b4b2ad-c3c3-4909-86b0-61aec164facf-kube-api-access-l4qmn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv\" (UID: \"10b4b2ad-c3c3-4909-86b0-61aec164facf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"
Jul 24 18:50:06.987762 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.987663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10b4b2ad-c3c3-4909-86b0-61aec164facf-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv\" (UID: \"10b4b2ad-c3c3-4909-86b0-61aec164facf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"
Jul 24 18:50:06.987762 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:06.987706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10b4b2ad-c3c3-4909-86b0-61aec164facf-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv\" (UID: \"10b4b2ad-c3c3-4909-86b0-61aec164facf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"
Jul 24 18:50:07.007722 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.007695    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"]
Jul 24 18:50:07.009185 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:50:07.009156    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9797f2d8_5332_4ea3_8033_422021d8bc0c.slice/crio-ef5bd22c9e0c4f0fc8ffdbc3a7214b1807881f4be10b2f20b4535ddce23300af WatchSource:0}: Error finding container ef5bd22c9e0c4f0fc8ffdbc3a7214b1807881f4be10b2f20b4535ddce23300af: Status 404 returned error can't find the container with id ef5bd22c9e0c4f0fc8ffdbc3a7214b1807881f4be10b2f20b4535ddce23300af
Jul 24 18:50:07.089125 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.089086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10b4b2ad-c3c3-4909-86b0-61aec164facf-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv\" (UID: \"10b4b2ad-c3c3-4909-86b0-61aec164facf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"
Jul 24 18:50:07.089125 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.089129    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10b4b2ad-c3c3-4909-86b0-61aec164facf-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv\" (UID: \"10b4b2ad-c3c3-4909-86b0-61aec164facf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"
Jul 24 18:50:07.089320 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.089167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4qmn\" (UniqueName: \"kubernetes.io/projected/10b4b2ad-c3c3-4909-86b0-61aec164facf-kube-api-access-l4qmn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv\" (UID: \"10b4b2ad-c3c3-4909-86b0-61aec164facf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"
Jul 24 18:50:07.089640 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.089617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10b4b2ad-c3c3-4909-86b0-61aec164facf-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv\" (UID: \"10b4b2ad-c3c3-4909-86b0-61aec164facf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"
Jul 24 18:50:07.089683 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.089620    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10b4b2ad-c3c3-4909-86b0-61aec164facf-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv\" (UID: \"10b4b2ad-c3c3-4909-86b0-61aec164facf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"
Jul 24 18:50:07.097381 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.097354    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4qmn\" (UniqueName: \"kubernetes.io/projected/10b4b2ad-c3c3-4909-86b0-61aec164facf-kube-api-access-l4qmn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv\" (UID: \"10b4b2ad-c3c3-4909-86b0-61aec164facf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"
Jul 24 18:50:07.269101 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.269027    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"
Jul 24 18:50:07.411648 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.411615    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"]
Jul 24 18:50:07.413774 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:50:07.413750    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10b4b2ad_c3c3_4909_86b0_61aec164facf.slice/crio-8c47c9a7a348e0ce6e0fde9714450db808be1832479373a546dfcac494024b31 WatchSource:0}: Error finding container 8c47c9a7a348e0ce6e0fde9714450db808be1832479373a546dfcac494024b31: Status 404 returned error can't find the container with id 8c47c9a7a348e0ce6e0fde9714450db808be1832479373a546dfcac494024b31
Jul 24 18:50:07.677322 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.677283    2563 generic.go:356] "Generic (PLEG): container finished" podID="9797f2d8-5332-4ea3-8033-422021d8bc0c" containerID="cd48cf45ec3cbd74bd4eca52c726849bf6489579048be6a4b9aed4ffc61393b4" exitCode=0
Jul 24 18:50:07.677787 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.677371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs" event={"ID":"9797f2d8-5332-4ea3-8033-422021d8bc0c","Type":"ContainerDied","Data":"cd48cf45ec3cbd74bd4eca52c726849bf6489579048be6a4b9aed4ffc61393b4"}
Jul 24 18:50:07.677787 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.677451    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs" event={"ID":"9797f2d8-5332-4ea3-8033-422021d8bc0c","Type":"ContainerStarted","Data":"ef5bd22c9e0c4f0fc8ffdbc3a7214b1807881f4be10b2f20b4535ddce23300af"}
Jul 24 18:50:07.679051 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.679030    2563 generic.go:356] "Generic (PLEG): container finished" podID="62188e01-a45c-4f10-bfba-24ecbf9c1000" containerID="ec172977cec1338c6eba205f45036a7fd8b1da4f9a74da9b4d5b6bed65897b85" exitCode=0
Jul 24 18:50:07.679172 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.679096    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv" event={"ID":"62188e01-a45c-4f10-bfba-24ecbf9c1000","Type":"ContainerDied","Data":"ec172977cec1338c6eba205f45036a7fd8b1da4f9a74da9b4d5b6bed65897b85"}
Jul 24 18:50:07.681204 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.681181    2563 generic.go:356] "Generic (PLEG): container finished" podID="511f5628-511a-4c8a-8d0b-88dc56223da3" containerID="060aefa88f43906702a9e22631265ddd575b0a3ff6d7e89a45ed25f118b8634f" exitCode=0
Jul 24 18:50:07.681306 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.681210    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f" event={"ID":"511f5628-511a-4c8a-8d0b-88dc56223da3","Type":"ContainerDied","Data":"060aefa88f43906702a9e22631265ddd575b0a3ff6d7e89a45ed25f118b8634f"}
Jul 24 18:50:07.682927 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.682902    2563 generic.go:356] "Generic (PLEG): container finished" podID="10b4b2ad-c3c3-4909-86b0-61aec164facf" containerID="070d98f7440565a9a8fe4f2efbacc4103e65a3cb9938124e5ac91fc91b5d3b61" exitCode=0
Jul 24 18:50:07.683025 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.682947    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv" event={"ID":"10b4b2ad-c3c3-4909-86b0-61aec164facf","Type":"ContainerDied","Data":"070d98f7440565a9a8fe4f2efbacc4103e65a3cb9938124e5ac91fc91b5d3b61"}
Jul 24 18:50:07.683025 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:07.682978    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv" event={"ID":"10b4b2ad-c3c3-4909-86b0-61aec164facf","Type":"ContainerStarted","Data":"8c47c9a7a348e0ce6e0fde9714450db808be1832479373a546dfcac494024b31"}
Jul 24 18:50:08.689740 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:08.689704    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs" event={"ID":"9797f2d8-5332-4ea3-8033-422021d8bc0c","Type":"ContainerStarted","Data":"f433cb92f948e6899543e29e03f1f2024e47a3224c7631109b84a927bae3ef5c"}
Jul 24 18:50:08.691614 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:08.691589    2563 generic.go:356] "Generic (PLEG): container finished" podID="62188e01-a45c-4f10-bfba-24ecbf9c1000" containerID="9beb9517f5795e28caee7f88eceba5ea96489bc36dc6c1bc1c53a043229b4f74" exitCode=0
Jul 24 18:50:08.691715 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:08.691664    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv" event={"ID":"62188e01-a45c-4f10-bfba-24ecbf9c1000","Type":"ContainerDied","Data":"9beb9517f5795e28caee7f88eceba5ea96489bc36dc6c1bc1c53a043229b4f74"}
Jul 24 18:50:08.694110 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:08.694079    2563 generic.go:356] "Generic (PLEG): container finished" podID="511f5628-511a-4c8a-8d0b-88dc56223da3" containerID="0f4505133c7a09522c2a1b3b5cfbca78f54077bb52f821cb3b35800a260fea0d" exitCode=0
Jul 24 18:50:08.694332 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:08.694163    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f" event={"ID":"511f5628-511a-4c8a-8d0b-88dc56223da3","Type":"ContainerDied","Data":"0f4505133c7a09522c2a1b3b5cfbca78f54077bb52f821cb3b35800a260fea0d"}
Jul 24 18:50:08.696231 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:08.696048    2563 generic.go:356] "Generic (PLEG): container finished" podID="10b4b2ad-c3c3-4909-86b0-61aec164facf" containerID="e653f4ce98ab45ed2d36fd2ea90a3b9788809ac150f31b4043228f8cfcd45a81" exitCode=0
Jul 24 18:50:08.696231 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:08.696107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv" event={"ID":"10b4b2ad-c3c3-4909-86b0-61aec164facf","Type":"ContainerDied","Data":"e653f4ce98ab45ed2d36fd2ea90a3b9788809ac150f31b4043228f8cfcd45a81"}
Jul 24 18:50:09.702555 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:09.702523    2563 generic.go:356] "Generic (PLEG): container finished" podID="9797f2d8-5332-4ea3-8033-422021d8bc0c" containerID="f433cb92f948e6899543e29e03f1f2024e47a3224c7631109b84a927bae3ef5c" exitCode=0
Jul 24 18:50:09.703016 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:09.702618    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs" event={"ID":"9797f2d8-5332-4ea3-8033-422021d8bc0c","Type":"ContainerDied","Data":"f433cb92f948e6899543e29e03f1f2024e47a3224c7631109b84a927bae3ef5c"}
Jul 24 18:50:09.705006 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:09.704983    2563 generic.go:356] "Generic (PLEG): container finished" podID="62188e01-a45c-4f10-bfba-24ecbf9c1000" containerID="e709584d59b70ae1ef54c1a26563e0f86467394a676ec535fcde2495e90092af" exitCode=0
Jul 24 18:50:09.705107 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:09.705021    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv" event={"ID":"62188e01-a45c-4f10-bfba-24ecbf9c1000","Type":"ContainerDied","Data":"e709584d59b70ae1ef54c1a26563e0f86467394a676ec535fcde2495e90092af"}
Jul 24 18:50:09.707118 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:09.707099    2563 generic.go:356] "Generic (PLEG): container finished" podID="10b4b2ad-c3c3-4909-86b0-61aec164facf" containerID="add0704a3a6f4535cb0d143b67f7a677cf8dc8c454a9062586a8af86f12b6ac8" exitCode=0
Jul 24 18:50:09.707223 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:09.707121    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv" event={"ID":"10b4b2ad-c3c3-4909-86b0-61aec164facf","Type":"ContainerDied","Data":"add0704a3a6f4535cb0d143b67f7a677cf8dc8c454a9062586a8af86f12b6ac8"}
Jul 24 18:50:09.839290 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:09.839267    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"
Jul 24 18:50:09.913570 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:09.913538    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/511f5628-511a-4c8a-8d0b-88dc56223da3-bundle\") pod \"511f5628-511a-4c8a-8d0b-88dc56223da3\" (UID: \"511f5628-511a-4c8a-8d0b-88dc56223da3\") "
Jul 24 18:50:09.913754 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:09.913592    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/511f5628-511a-4c8a-8d0b-88dc56223da3-util\") pod \"511f5628-511a-4c8a-8d0b-88dc56223da3\" (UID: \"511f5628-511a-4c8a-8d0b-88dc56223da3\") "
Jul 24 18:50:09.913754 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:09.913626    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmzzw\" (UniqueName: \"kubernetes.io/projected/511f5628-511a-4c8a-8d0b-88dc56223da3-kube-api-access-dmzzw\") pod \"511f5628-511a-4c8a-8d0b-88dc56223da3\" (UID: \"511f5628-511a-4c8a-8d0b-88dc56223da3\") "
Jul 24 18:50:09.914111 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:09.914085    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/511f5628-511a-4c8a-8d0b-88dc56223da3-bundle" (OuterVolumeSpecName: "bundle") pod "511f5628-511a-4c8a-8d0b-88dc56223da3" (UID: "511f5628-511a-4c8a-8d0b-88dc56223da3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:50:09.916132 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:09.916105    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/511f5628-511a-4c8a-8d0b-88dc56223da3-kube-api-access-dmzzw" (OuterVolumeSpecName: "kube-api-access-dmzzw") pod "511f5628-511a-4c8a-8d0b-88dc56223da3" (UID: "511f5628-511a-4c8a-8d0b-88dc56223da3"). InnerVolumeSpecName "kube-api-access-dmzzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:50:09.918984 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:09.918944    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/511f5628-511a-4c8a-8d0b-88dc56223da3-util" (OuterVolumeSpecName: "util") pod "511f5628-511a-4c8a-8d0b-88dc56223da3" (UID: "511f5628-511a-4c8a-8d0b-88dc56223da3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:50:10.014732 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.014645    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/511f5628-511a-4c8a-8d0b-88dc56223da3-bundle\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:50:10.014732 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.014677    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/511f5628-511a-4c8a-8d0b-88dc56223da3-util\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:50:10.014732 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.014686    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmzzw\" (UniqueName: \"kubernetes.io/projected/511f5628-511a-4c8a-8d0b-88dc56223da3-kube-api-access-dmzzw\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:50:10.714515 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.714480    2563 generic.go:356] "Generic (PLEG): container finished" podID="9797f2d8-5332-4ea3-8033-422021d8bc0c" containerID="36ea7eb3eb774691012c5437619f12ee294dec044d1920dac92f89c93a02f3a7" exitCode=0
Jul 24 18:50:10.714942 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.714564    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs" event={"ID":"9797f2d8-5332-4ea3-8033-422021d8bc0c","Type":"ContainerDied","Data":"36ea7eb3eb774691012c5437619f12ee294dec044d1920dac92f89c93a02f3a7"}
Jul 24 18:50:10.716478 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.716461    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f"
Jul 24 18:50:10.716520 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.716487    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f" event={"ID":"511f5628-511a-4c8a-8d0b-88dc56223da3","Type":"ContainerDied","Data":"dd3541069973bdc742f799924efc790a2c6e3287f087e1bcee1938d12a395e20"}
Jul 24 18:50:10.716562 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.716526    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd3541069973bdc742f799924efc790a2c6e3287f087e1bcee1938d12a395e20"
Jul 24 18:50:10.845772 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.845749    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"
Jul 24 18:50:10.879578 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.879554    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"
Jul 24 18:50:10.922065 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.922035    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10b4b2ad-c3c3-4909-86b0-61aec164facf-bundle\") pod \"10b4b2ad-c3c3-4909-86b0-61aec164facf\" (UID: \"10b4b2ad-c3c3-4909-86b0-61aec164facf\") "
Jul 24 18:50:10.922257 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.922098    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62188e01-a45c-4f10-bfba-24ecbf9c1000-util\") pod \"62188e01-a45c-4f10-bfba-24ecbf9c1000\" (UID: \"62188e01-a45c-4f10-bfba-24ecbf9c1000\") "
Jul 24 18:50:10.922257 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.922126    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgqv5\" (UniqueName: \"kubernetes.io/projected/62188e01-a45c-4f10-bfba-24ecbf9c1000-kube-api-access-mgqv5\") pod \"62188e01-a45c-4f10-bfba-24ecbf9c1000\" (UID: \"62188e01-a45c-4f10-bfba-24ecbf9c1000\") "
Jul 24 18:50:10.922257 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.922143    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10b4b2ad-c3c3-4909-86b0-61aec164facf-util\") pod \"10b4b2ad-c3c3-4909-86b0-61aec164facf\" (UID: \"10b4b2ad-c3c3-4909-86b0-61aec164facf\") "
Jul 24 18:50:10.922257 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.922167    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4qmn\" (UniqueName: \"kubernetes.io/projected/10b4b2ad-c3c3-4909-86b0-61aec164facf-kube-api-access-l4qmn\") pod \"10b4b2ad-c3c3-4909-86b0-61aec164facf\" (UID: \"10b4b2ad-c3c3-4909-86b0-61aec164facf\") "
Jul 24 18:50:10.922257 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.922209    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62188e01-a45c-4f10-bfba-24ecbf9c1000-bundle\") pod \"62188e01-a45c-4f10-bfba-24ecbf9c1000\" (UID: \"62188e01-a45c-4f10-bfba-24ecbf9c1000\") "
Jul 24 18:50:10.922812 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.922778    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62188e01-a45c-4f10-bfba-24ecbf9c1000-bundle" (OuterVolumeSpecName: "bundle") pod "62188e01-a45c-4f10-bfba-24ecbf9c1000" (UID: "62188e01-a45c-4f10-bfba-24ecbf9c1000"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:50:10.922934 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.922901    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10b4b2ad-c3c3-4909-86b0-61aec164facf-bundle" (OuterVolumeSpecName: "bundle") pod "10b4b2ad-c3c3-4909-86b0-61aec164facf" (UID: "10b4b2ad-c3c3-4909-86b0-61aec164facf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:50:10.924373 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.924347    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10b4b2ad-c3c3-4909-86b0-61aec164facf-kube-api-access-l4qmn" (OuterVolumeSpecName: "kube-api-access-l4qmn") pod "10b4b2ad-c3c3-4909-86b0-61aec164facf" (UID: "10b4b2ad-c3c3-4909-86b0-61aec164facf"). InnerVolumeSpecName "kube-api-access-l4qmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:50:10.924670 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.924643    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62188e01-a45c-4f10-bfba-24ecbf9c1000-kube-api-access-mgqv5" (OuterVolumeSpecName: "kube-api-access-mgqv5") pod "62188e01-a45c-4f10-bfba-24ecbf9c1000" (UID: "62188e01-a45c-4f10-bfba-24ecbf9c1000"). InnerVolumeSpecName "kube-api-access-mgqv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:50:10.927957 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.927922    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10b4b2ad-c3c3-4909-86b0-61aec164facf-util" (OuterVolumeSpecName: "util") pod "10b4b2ad-c3c3-4909-86b0-61aec164facf" (UID: "10b4b2ad-c3c3-4909-86b0-61aec164facf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:50:10.929911 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:10.929882    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62188e01-a45c-4f10-bfba-24ecbf9c1000-util" (OuterVolumeSpecName: "util") pod "62188e01-a45c-4f10-bfba-24ecbf9c1000" (UID: "62188e01-a45c-4f10-bfba-24ecbf9c1000"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:50:11.023194 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.023096    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62188e01-a45c-4f10-bfba-24ecbf9c1000-bundle\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:50:11.023194 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.023130    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10b4b2ad-c3c3-4909-86b0-61aec164facf-bundle\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:50:11.023194 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.023142    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62188e01-a45c-4f10-bfba-24ecbf9c1000-util\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:50:11.023194 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.023154    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgqv5\" (UniqueName: \"kubernetes.io/projected/62188e01-a45c-4f10-bfba-24ecbf9c1000-kube-api-access-mgqv5\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:50:11.023194 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.023168    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10b4b2ad-c3c3-4909-86b0-61aec164facf-util\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:50:11.023194 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.023181    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4qmn\" (UniqueName: \"kubernetes.io/projected/10b4b2ad-c3c3-4909-86b0-61aec164facf-kube-api-access-l4qmn\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:50:11.722911 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.722869    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv"
Jul 24 18:50:11.722911 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.722875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv" event={"ID":"62188e01-a45c-4f10-bfba-24ecbf9c1000","Type":"ContainerDied","Data":"97c98bb0f50bee1c7b102f93fa6274e3cf5eec6a9366f600418e3010e04cf6d5"}
Jul 24 18:50:11.723543 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.722943    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97c98bb0f50bee1c7b102f93fa6274e3cf5eec6a9366f600418e3010e04cf6d5"
Jul 24 18:50:11.724781 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.724761    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv"
Jul 24 18:50:11.724781 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.724773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv" event={"ID":"10b4b2ad-c3c3-4909-86b0-61aec164facf","Type":"ContainerDied","Data":"8c47c9a7a348e0ce6e0fde9714450db808be1832479373a546dfcac494024b31"}
Jul 24 18:50:11.724919 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.724815    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c47c9a7a348e0ce6e0fde9714450db808be1832479373a546dfcac494024b31"
Jul 24 18:50:11.849926 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.849905    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"
Jul 24 18:50:11.930558 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.930525    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spqq4\" (UniqueName: \"kubernetes.io/projected/9797f2d8-5332-4ea3-8033-422021d8bc0c-kube-api-access-spqq4\") pod \"9797f2d8-5332-4ea3-8033-422021d8bc0c\" (UID: \"9797f2d8-5332-4ea3-8033-422021d8bc0c\") "
Jul 24 18:50:11.930558 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.930568    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9797f2d8-5332-4ea3-8033-422021d8bc0c-util\") pod \"9797f2d8-5332-4ea3-8033-422021d8bc0c\" (UID: \"9797f2d8-5332-4ea3-8033-422021d8bc0c\") "
Jul 24 18:50:11.930813 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.930617    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9797f2d8-5332-4ea3-8033-422021d8bc0c-bundle\") pod \"9797f2d8-5332-4ea3-8033-422021d8bc0c\" (UID: \"9797f2d8-5332-4ea3-8033-422021d8bc0c\") "
Jul 24 18:50:11.931058 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.931035    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9797f2d8-5332-4ea3-8033-422021d8bc0c-bundle" (OuterVolumeSpecName: "bundle") pod "9797f2d8-5332-4ea3-8033-422021d8bc0c" (UID: "9797f2d8-5332-4ea3-8033-422021d8bc0c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:50:11.932937 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.932913    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9797f2d8-5332-4ea3-8033-422021d8bc0c-kube-api-access-spqq4" (OuterVolumeSpecName: "kube-api-access-spqq4") pod "9797f2d8-5332-4ea3-8033-422021d8bc0c" (UID: "9797f2d8-5332-4ea3-8033-422021d8bc0c"). InnerVolumeSpecName "kube-api-access-spqq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:50:11.935756 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:11.935721    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9797f2d8-5332-4ea3-8033-422021d8bc0c-util" (OuterVolumeSpecName: "util") pod "9797f2d8-5332-4ea3-8033-422021d8bc0c" (UID: "9797f2d8-5332-4ea3-8033-422021d8bc0c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:50:12.031637 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:12.031548    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spqq4\" (UniqueName: \"kubernetes.io/projected/9797f2d8-5332-4ea3-8033-422021d8bc0c-kube-api-access-spqq4\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:50:12.031637 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:12.031578    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9797f2d8-5332-4ea3-8033-422021d8bc0c-util\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:50:12.031637 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:12.031587    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9797f2d8-5332-4ea3-8033-422021d8bc0c-bundle\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 18:50:12.731322 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:12.731296    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs"
Jul 24 18:50:12.731728 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:12.731296    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs" event={"ID":"9797f2d8-5332-4ea3-8033-422021d8bc0c","Type":"ContainerDied","Data":"ef5bd22c9e0c4f0fc8ffdbc3a7214b1807881f4be10b2f20b4535ddce23300af"}
Jul 24 18:50:12.731728 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:50:12.731428    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef5bd22c9e0c4f0fc8ffdbc3a7214b1807881f4be10b2f20b4535ddce23300af"
Jul 24 18:51:17.772004 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.771968    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772227    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9797f2d8-5332-4ea3-8033-422021d8bc0c" containerName="extract"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772238    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9797f2d8-5332-4ea3-8033-422021d8bc0c" containerName="extract"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772253    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62188e01-a45c-4f10-bfba-24ecbf9c1000" containerName="extract"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772258    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="62188e01-a45c-4f10-bfba-24ecbf9c1000" containerName="extract"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772264    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10b4b2ad-c3c3-4909-86b0-61aec164facf" containerName="util"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772270    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="10b4b2ad-c3c3-4909-86b0-61aec164facf" containerName="util"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772280    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62188e01-a45c-4f10-bfba-24ecbf9c1000" containerName="util"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772286    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="62188e01-a45c-4f10-bfba-24ecbf9c1000" containerName="util"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772292    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9797f2d8-5332-4ea3-8033-422021d8bc0c" containerName="util"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772296    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9797f2d8-5332-4ea3-8033-422021d8bc0c" containerName="util"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772303    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62188e01-a45c-4f10-bfba-24ecbf9c1000" containerName="pull"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772307    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="62188e01-a45c-4f10-bfba-24ecbf9c1000" containerName="pull"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772316    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10b4b2ad-c3c3-4909-86b0-61aec164facf" containerName="pull"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772321    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="10b4b2ad-c3c3-4909-86b0-61aec164facf" containerName="pull"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772328    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9797f2d8-5332-4ea3-8033-422021d8bc0c" containerName="pull"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772333    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9797f2d8-5332-4ea3-8033-422021d8bc0c" containerName="pull"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772338    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10b4b2ad-c3c3-4909-86b0-61aec164facf" containerName="extract"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772344    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="10b4b2ad-c3c3-4909-86b0-61aec164facf" containerName="extract"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772350    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="511f5628-511a-4c8a-8d0b-88dc56223da3" containerName="util"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772354    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="511f5628-511a-4c8a-8d0b-88dc56223da3" containerName="util"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772358    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="511f5628-511a-4c8a-8d0b-88dc56223da3" containerName="pull"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772363    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="511f5628-511a-4c8a-8d0b-88dc56223da3" containerName="pull"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772368    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="511f5628-511a-4c8a-8d0b-88dc56223da3" containerName="extract"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772373    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="511f5628-511a-4c8a-8d0b-88dc56223da3" containerName="extract"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772437    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="62188e01-a45c-4f10-bfba-24ecbf9c1000" containerName="extract"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772445    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9797f2d8-5332-4ea3-8033-422021d8bc0c" containerName="extract"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772452    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="10b4b2ad-c3c3-4909-86b0-61aec164facf" containerName="extract"
Jul 24 18:51:17.772503 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.772459    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="511f5628-511a-4c8a-8d0b-88dc56223da3" containerName="extract"
Jul 24 18:51:17.775040 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.775025    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"
Jul 24 18:51:17.777334 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.777306    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jwg7r\""
Jul 24 18:51:17.777461 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.777435    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 18:51:17.778235 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.778218    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 18:51:17.778290 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.778226    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 18:51:17.783186 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.783165    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:51:17.807613 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.807586    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:51:17.858241 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.858205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b8246d36-7da8-4e55-a471-0232cc4eed49-config-file\") pod \"limitador-limitador-54d6d988d5-q4bw8\" (UID: \"b8246d36-7da8-4e55-a471-0232cc4eed49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"
Jul 24 18:51:17.858432 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.858259    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp2pf\" (UniqueName: \"kubernetes.io/projected/b8246d36-7da8-4e55-a471-0232cc4eed49-kube-api-access-hp2pf\") pod \"limitador-limitador-54d6d988d5-q4bw8\" (UID: \"b8246d36-7da8-4e55-a471-0232cc4eed49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"
Jul 24 18:51:17.958984 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.958946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b8246d36-7da8-4e55-a471-0232cc4eed49-config-file\") pod \"limitador-limitador-54d6d988d5-q4bw8\" (UID: \"b8246d36-7da8-4e55-a471-0232cc4eed49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"
Jul 24 18:51:17.958984 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.958990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp2pf\" (UniqueName: \"kubernetes.io/projected/b8246d36-7da8-4e55-a471-0232cc4eed49-kube-api-access-hp2pf\") pod \"limitador-limitador-54d6d988d5-q4bw8\" (UID: \"b8246d36-7da8-4e55-a471-0232cc4eed49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"
Jul 24 18:51:17.959607 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.959586    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b8246d36-7da8-4e55-a471-0232cc4eed49-config-file\") pod \"limitador-limitador-54d6d988d5-q4bw8\" (UID: \"b8246d36-7da8-4e55-a471-0232cc4eed49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"
Jul 24 18:51:17.966881 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:17.966856    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp2pf\" (UniqueName: \"kubernetes.io/projected/b8246d36-7da8-4e55-a471-0232cc4eed49-kube-api-access-hp2pf\") pod \"limitador-limitador-54d6d988d5-q4bw8\" (UID: \"b8246d36-7da8-4e55-a471-0232cc4eed49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"
Jul 24 18:51:18.085578 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:18.085545    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"
Jul 24 18:51:18.219684 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:18.219658    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:51:18.221906 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:51:18.221870    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8246d36_7da8_4e55_a471_0232cc4eed49.slice/crio-ade87a9de79dfbb9c9b8d3ec0af0647e4e0b7d44d74e6ddc3badec563c93857e WatchSource:0}: Error finding container ade87a9de79dfbb9c9b8d3ec0af0647e4e0b7d44d74e6ddc3badec563c93857e: Status 404 returned error can't find the container with id ade87a9de79dfbb9c9b8d3ec0af0647e4e0b7d44d74e6ddc3badec563c93857e
Jul 24 18:51:19.046356 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:19.046308    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-q4bw8" event={"ID":"b8246d36-7da8-4e55-a471-0232cc4eed49","Type":"ContainerStarted","Data":"ade87a9de79dfbb9c9b8d3ec0af0647e4e0b7d44d74e6ddc3badec563c93857e"}
Jul 24 18:51:21.057950 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:21.057920    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-q4bw8" event={"ID":"b8246d36-7da8-4e55-a471-0232cc4eed49","Type":"ContainerStarted","Data":"34e63875350735db599a86c644b6bb2982bd24239511037ded9017e8ecfc3b08"}
Jul 24 18:51:21.058354 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:21.058066    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"
Jul 24 18:51:21.100774 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:21.100723    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-q4bw8" podStartSLOduration=1.354080113 podStartE2EDuration="4.100706324s" totalImagesPullingTime="2.746626211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:51:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:51:21.099364404 +0000 UTC m=+419.490315881" watchObservedRunningTime="2026-07-24 18:51:21.100706324 +0000 UTC m=+419.491657801"
Jul 24 18:51:32.067102 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:51:32.067066    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"
Jul 24 18:52:27.857558 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:27.857525    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:52:44.867687 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:44.867644    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:52:52.950330 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:52.950294    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:52:58.467401 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.467353    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"]
Jul 24 18:52:58.470465 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.470444    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.472301 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.472275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2308a1d7-63df-4d83-8f98-0536d233d4ba-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.472440 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.472325    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkphc\" (UniqueName: \"kubernetes.io/projected/2308a1d7-63df-4d83-8f98-0536d233d4ba-kube-api-access-fkphc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.472440 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.472406    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2308a1d7-63df-4d83-8f98-0536d233d4ba-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.472530 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.472441    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2308a1d7-63df-4d83-8f98-0536d233d4ba-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.472530 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.472459    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2308a1d7-63df-4d83-8f98-0536d233d4ba-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.472530 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.472482    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2308a1d7-63df-4d83-8f98-0536d233d4ba-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.472977 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.472955    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 24 18:52:58.473057 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.472965    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 18:52:58.473057 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.472992    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 18:52:58.473057 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.473028    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tnr97\""
Jul 24 18:52:58.480615 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.480593    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"]
Jul 24 18:52:58.573493 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.573459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2308a1d7-63df-4d83-8f98-0536d233d4ba-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.573493 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.573496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2308a1d7-63df-4d83-8f98-0536d233d4ba-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.573744 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.573514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2308a1d7-63df-4d83-8f98-0536d233d4ba-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.573744 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.573530    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2308a1d7-63df-4d83-8f98-0536d233d4ba-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.573744 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.573578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkphc\" (UniqueName: \"kubernetes.io/projected/2308a1d7-63df-4d83-8f98-0536d233d4ba-kube-api-access-fkphc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.573744 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.573613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2308a1d7-63df-4d83-8f98-0536d233d4ba-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.573980 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.573938    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2308a1d7-63df-4d83-8f98-0536d233d4ba-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.574025 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.573978    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2308a1d7-63df-4d83-8f98-0536d233d4ba-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.574025 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.574006    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2308a1d7-63df-4d83-8f98-0536d233d4ba-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.576011 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.575981    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2308a1d7-63df-4d83-8f98-0536d233d4ba-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.576157 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.576144    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2308a1d7-63df-4d83-8f98-0536d233d4ba-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.581750 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.581725    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkphc\" (UniqueName: \"kubernetes.io/projected/2308a1d7-63df-4d83-8f98-0536d233d4ba-kube-api-access-fkphc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fjph6\" (UID: \"2308a1d7-63df-4d83-8f98-0536d233d4ba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.781123 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.781032    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:52:58.916321 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:58.916295    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"]
Jul 24 18:52:58.918236 ip-10-0-141-213 kubenswrapper[2563]: W0724 18:52:58.918208    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2308a1d7_63df_4d83_8f98_0536d233d4ba.slice/crio-6590a78e9c555b779a0f93c0b0c844fcd1e5e3d71df26100bb21baf440885392 WatchSource:0}: Error finding container 6590a78e9c555b779a0f93c0b0c844fcd1e5e3d71df26100bb21baf440885392: Status 404 returned error can't find the container with id 6590a78e9c555b779a0f93c0b0c844fcd1e5e3d71df26100bb21baf440885392
Jul 24 18:52:59.513831 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:59.513799    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6" event={"ID":"2308a1d7-63df-4d83-8f98-0536d233d4ba","Type":"ContainerStarted","Data":"6590a78e9c555b779a0f93c0b0c844fcd1e5e3d71df26100bb21baf440885392"}
Jul 24 18:52:59.750919 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:52:59.750876    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:53:00.519870 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:53:00.519837    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6" event={"ID":"2308a1d7-63df-4d83-8f98-0536d233d4ba","Type":"ContainerStarted","Data":"327ec819821e5f342d039422e303ab1ea305625b0d2cbaa9f28bd4db4a6276cf"}
Jul 24 18:53:00.539730 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:53:00.539668    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6" podStartSLOduration=1.558348914 podStartE2EDuration="2.539647123s" totalImagesPullingTime="981.298209ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:52:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:53:00.536498468 +0000 UTC m=+518.927449956" watchObservedRunningTime="2026-07-24 18:53:00.539647123 +0000 UTC m=+518.930598602"
Jul 24 18:53:06.756548 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:53:06.756513    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:53:08.781563 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:53:08.781526    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:53:08.782046 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:53:08.781614    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:53:08.794333 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:53:08.794303    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:53:09.572833 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:53:09.572808    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fjph6"
Jul 24 18:53:13.847416 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:53:13.847316    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:53:22.749850 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:53:22.749812    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:54:22.072437 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:54:22.072410    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 18:54:22.073206 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:54:22.073178    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 18:54:34.652873 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:54:34.652825    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:54:44.045357 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:54:44.045274    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:54:55.150699 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:54:55.150666    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:56:36.347483 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:56:36.347445    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:56:48.657712 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:56:48.657677    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:57:22.451647 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:57:22.451605    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:57:31.744354 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:57:31.744320    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:57:42.447722 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:57:42.447639    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:57:50.945249 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:57:50.945213    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:58:01.759554 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:58:01.759520    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:58:11.247124 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:58:11.247092    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:58:22.351327 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:58:22.351296    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:58:32.551535 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:58:32.551496    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:58:54.451784 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:58:54.451751    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:59:07.352496 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:59:07.352459    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:59:22.103264 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:59:22.103181    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 18:59:22.106701 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:59:22.106677    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 18:59:29.460411 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:59:29.460364    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:59:39.751556 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:59:39.751523    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 18:59:49.154191 ip-10-0-141-213 kubenswrapper[2563]: I0724 18:59:49.154154    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:00:05.564642 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:00:05.564605    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:00:14.957688 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:00:14.957652    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:00:24.562647 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:00:24.562615    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:00:34.409678 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:00:34.409642    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:00:43.253294 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:00:43.253215    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:03:39.659813 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:03:39.659779    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:03:49.846030 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:03:49.845997    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:03:59.948092 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:03:59.948050    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:04:08.255718 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:04:08.255687    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:04:17.948323 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:04:17.948280    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:04:22.135161 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:04:22.135133    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 19:04:22.139114 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:04:22.139095    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 19:04:26.958464 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:04:26.958425    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:04:35.148895 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:04:35.148858    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:04:44.659759 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:04:44.659727    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:04:53.252699 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:04:53.252654    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:05:04.443198 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:04.443144    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:05:25.051178 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:25.051100    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:05:34.855500 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:34.855463    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:05:36.976234 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:36.976197    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"]
Jul 24 19:05:36.979617 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:36.979592    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:36.982086 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:36.982070    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-87181f2e-kserve-self-signed-certs\""
Jul 24 19:05:36.998475 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:36.998455    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"]
Jul 24 19:05:37.080705 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.080672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-dshm\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.080705 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.080706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71f12591-0ecd-4e43-9732-7e84eb3f0819-tls-certs\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.080896 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.080727    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-model-cache\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.080896 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.080791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-tmp-dir\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.080896 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.080836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq4bm\" (UniqueName: \"kubernetes.io/projected/71f12591-0ecd-4e43-9732-7e84eb3f0819-kube-api-access-kq4bm\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.080896 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.080853    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-home\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.182034 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.182004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq4bm\" (UniqueName: \"kubernetes.io/projected/71f12591-0ecd-4e43-9732-7e84eb3f0819-kube-api-access-kq4bm\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.182222 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.182040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-home\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.182222 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.182070    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-dshm\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.182222 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.182087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71f12591-0ecd-4e43-9732-7e84eb3f0819-tls-certs\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.182222 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.182125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-model-cache\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.182222 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.182186    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-tmp-dir\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.182529 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.182432    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-home\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.182583 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.182540    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-model-cache\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.182623 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.182608    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-tmp-dir\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.184625 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.184606    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-dshm\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.184889 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.184869    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71f12591-0ecd-4e43-9732-7e84eb3f0819-tls-certs\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.189658 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.189637    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq4bm\" (UniqueName: \"kubernetes.io/projected/71f12591-0ecd-4e43-9732-7e84eb3f0819-kube-api-access-kq4bm\") pod \"e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") " pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.288428 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.288326    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:37.421816 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:37.421792    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"]
Jul 24 19:05:37.424565 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:05:37.424534    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71f12591_0ecd_4e43_9732_7e84eb3f0819.slice/crio-44fcff609ca944989fc2ba25829fc5a3e7303dc7ca4ec0d422e710981865ef9c WatchSource:0}: Error finding container 44fcff609ca944989fc2ba25829fc5a3e7303dc7ca4ec0d422e710981865ef9c: Status 404 returned error can't find the container with id 44fcff609ca944989fc2ba25829fc5a3e7303dc7ca4ec0d422e710981865ef9c
Jul 24 19:05:38.103362 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:38.103326    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p" event={"ID":"71f12591-0ecd-4e43-9732-7e84eb3f0819","Type":"ContainerStarted","Data":"0546b926fff44ee10d4ecfff453d155122058cbc710a93b0eaee4400432cdcc6"}
Jul 24 19:05:38.103362 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:38.103373    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p" event={"ID":"71f12591-0ecd-4e43-9732-7e84eb3f0819","Type":"ContainerStarted","Data":"44fcff609ca944989fc2ba25829fc5a3e7303dc7ca4ec0d422e710981865ef9c"}
Jul 24 19:05:38.125264 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:38.125218    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p" podStartSLOduration=2.125201333 podStartE2EDuration="2.125201333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:05:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:05:38.124110266 +0000 UTC m=+1276.515061742" watchObservedRunningTime="2026-07-24 19:05:38.125201333 +0000 UTC m=+1276.516152810"
Jul 24 19:05:47.288879 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:47.288784    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:47.288879 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:47.288844    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:47.301606 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:47.301579    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:48.166327 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:48.166300    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:05:49.048258 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:05:49.048222    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:06:05.216592 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:05.216557    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"]
Jul 24 19:06:09.653941 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:09.653906    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:06:20.236178 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.236116    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p" podUID="71f12591-0ecd-4e43-9732-7e84eb3f0819" containerName="main" containerID="cri-o://0546b926fff44ee10d4ecfff453d155122058cbc710a93b0eaee4400432cdcc6" gracePeriod=45
Jul 24 19:06:20.484222 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.484198    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:06:20.520344 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.520276    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71f12591-0ecd-4e43-9732-7e84eb3f0819-tls-certs\") pod \"71f12591-0ecd-4e43-9732-7e84eb3f0819\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") "
Jul 24 19:06:20.520344 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.520314    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-dshm\") pod \"71f12591-0ecd-4e43-9732-7e84eb3f0819\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") "
Jul 24 19:06:20.520344 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.520344    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-home\") pod \"71f12591-0ecd-4e43-9732-7e84eb3f0819\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") "
Jul 24 19:06:20.520603 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.520446    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-model-cache\") pod \"71f12591-0ecd-4e43-9732-7e84eb3f0819\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") "
Jul 24 19:06:20.520603 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.520488    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-tmp-dir\") pod \"71f12591-0ecd-4e43-9732-7e84eb3f0819\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") "
Jul 24 19:06:20.520603 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.520517    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kq4bm\" (UniqueName: \"kubernetes.io/projected/71f12591-0ecd-4e43-9732-7e84eb3f0819-kube-api-access-kq4bm\") pod \"71f12591-0ecd-4e43-9732-7e84eb3f0819\" (UID: \"71f12591-0ecd-4e43-9732-7e84eb3f0819\") "
Jul 24 19:06:20.520764 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.520600    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-home" (OuterVolumeSpecName: "home") pod "71f12591-0ecd-4e43-9732-7e84eb3f0819" (UID: "71f12591-0ecd-4e43-9732-7e84eb3f0819"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:06:20.520764 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.520718    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-model-cache" (OuterVolumeSpecName: "model-cache") pod "71f12591-0ecd-4e43-9732-7e84eb3f0819" (UID: "71f12591-0ecd-4e43-9732-7e84eb3f0819"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:06:20.520862 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.520822    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "71f12591-0ecd-4e43-9732-7e84eb3f0819" (UID: "71f12591-0ecd-4e43-9732-7e84eb3f0819"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:06:20.520862 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.520842    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-home\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:06:20.520960 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.520862    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-model-cache\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:06:20.522652 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.522620    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-dshm" (OuterVolumeSpecName: "dshm") pod "71f12591-0ecd-4e43-9732-7e84eb3f0819" (UID: "71f12591-0ecd-4e43-9732-7e84eb3f0819"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:06:20.522761 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.522651    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71f12591-0ecd-4e43-9732-7e84eb3f0819-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "71f12591-0ecd-4e43-9732-7e84eb3f0819" (UID: "71f12591-0ecd-4e43-9732-7e84eb3f0819"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:06:20.522761 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.522697    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71f12591-0ecd-4e43-9732-7e84eb3f0819-kube-api-access-kq4bm" (OuterVolumeSpecName: "kube-api-access-kq4bm") pod "71f12591-0ecd-4e43-9732-7e84eb3f0819" (UID: "71f12591-0ecd-4e43-9732-7e84eb3f0819"). InnerVolumeSpecName "kube-api-access-kq4bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:06:20.621500 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.621470    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71f12591-0ecd-4e43-9732-7e84eb3f0819-tls-certs\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:06:20.621500 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.621494    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-dshm\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:06:20.621667 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.621521    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71f12591-0ecd-4e43-9732-7e84eb3f0819-tmp-dir\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:06:20.621667 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:20.621530    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kq4bm\" (UniqueName: \"kubernetes.io/projected/71f12591-0ecd-4e43-9732-7e84eb3f0819-kube-api-access-kq4bm\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:06:21.315870 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:21.315829    2563 generic.go:356] "Generic (PLEG): container finished" podID="71f12591-0ecd-4e43-9732-7e84eb3f0819" containerID="0546b926fff44ee10d4ecfff453d155122058cbc710a93b0eaee4400432cdcc6" exitCode=0
Jul 24 19:06:21.316292 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:21.315885    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"
Jul 24 19:06:21.316292 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:21.315906    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p" event={"ID":"71f12591-0ecd-4e43-9732-7e84eb3f0819","Type":"ContainerDied","Data":"0546b926fff44ee10d4ecfff453d155122058cbc710a93b0eaee4400432cdcc6"}
Jul 24 19:06:21.316292 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:21.315951    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p" event={"ID":"71f12591-0ecd-4e43-9732-7e84eb3f0819","Type":"ContainerDied","Data":"44fcff609ca944989fc2ba25829fc5a3e7303dc7ca4ec0d422e710981865ef9c"}
Jul 24 19:06:21.316292 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:21.315968    2563 scope.go:117] "RemoveContainer" containerID="0546b926fff44ee10d4ecfff453d155122058cbc710a93b0eaee4400432cdcc6"
Jul 24 19:06:21.325804 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:21.325783    2563 scope.go:117] "RemoveContainer" containerID="0546b926fff44ee10d4ecfff453d155122058cbc710a93b0eaee4400432cdcc6"
Jul 24 19:06:21.326078 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:06:21.326059    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0546b926fff44ee10d4ecfff453d155122058cbc710a93b0eaee4400432cdcc6\": container with ID starting with 0546b926fff44ee10d4ecfff453d155122058cbc710a93b0eaee4400432cdcc6 not found: ID does not exist" containerID="0546b926fff44ee10d4ecfff453d155122058cbc710a93b0eaee4400432cdcc6"
Jul 24 19:06:21.326180 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:21.326085    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0546b926fff44ee10d4ecfff453d155122058cbc710a93b0eaee4400432cdcc6"} err="failed to get container status \"0546b926fff44ee10d4ecfff453d155122058cbc710a93b0eaee4400432cdcc6\": rpc error: code = NotFound desc = could not find container \"0546b926fff44ee10d4ecfff453d155122058cbc710a93b0eaee4400432cdcc6\": container with ID starting with 0546b926fff44ee10d4ecfff453d155122058cbc710a93b0eaee4400432cdcc6 not found: ID does not exist"
Jul 24 19:06:21.335798 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:21.335774    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"]
Jul 24 19:06:21.340112 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:21.340093    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-87181f2e-kserve-6cdb7bf86c-6m28p"]
Jul 24 19:06:22.204149 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:22.204117    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71f12591-0ecd-4e43-9732-7e84eb3f0819" path="/var/lib/kubelet/pods/71f12591-0ecd-4e43-9732-7e84eb3f0819/volumes"
Jul 24 19:06:23.350918 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:23.350888    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:06:30.252689 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:30.252652    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:06:39.558070 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:39.558030    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:06:47.864720 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:47.864688    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:06:57.683337 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:06:57.683298    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:07:06.954958 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:07:06.954920    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:07:17.750680 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:07:17.750649    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:07:26.456799 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:07:26.456761    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:07:35.851573 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:07:35.851530    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:07:48.752294 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:07:48.752256    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:07:57.458357 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:07:57.458318    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:08:07.250814 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:08:07.250770    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:08:19.349954 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:08:19.349861    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:08:28.752666 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:08:28.752632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:08:34.246942 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:08:34.246906    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:08:43.067905 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:08:43.067871    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:08:54.654766 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:08:54.654725    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:09:05.257561 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:05.257525    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:09:18.350861 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:18.350829    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:09:22.166466 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:22.166438    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 19:09:22.169977 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:22.169954    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 19:09:31.150613 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:31.150576    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:09:43.950126 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:43.950040    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:09:52.213967 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.213935    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"]
Jul 24 19:09:52.216195 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.214201    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71f12591-0ecd-4e43-9732-7e84eb3f0819" containerName="main"
Jul 24 19:09:52.216195 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.214212    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="71f12591-0ecd-4e43-9732-7e84eb3f0819" containerName="main"
Jul 24 19:09:52.216195 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.214266    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="71f12591-0ecd-4e43-9732-7e84eb3f0819" containerName="main"
Jul 24 19:09:52.217044 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.217030    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:09:52.220292 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.220269    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 19:09:52.220989 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.220963    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cvjqn\""
Jul 24 19:09:52.220989 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.220980    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d198c699-serving-cert\""
Jul 24 19:09:52.221129 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.221030    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 19:09:52.232306 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.232284    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"]
Jul 24 19:09:52.376636 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.376601    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-maas-api-tls\") pod \"maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn\" (UID: \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:09:52.376828 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.376743    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-openshift-service-ca\") pod \"maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn\" (UID: \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:09:52.376937 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.376853    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwsps\" (UniqueName: \"kubernetes.io/projected/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-kube-api-access-dwsps\") pod \"maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn\" (UID: \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:09:52.477333 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.477251    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwsps\" (UniqueName: \"kubernetes.io/projected/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-kube-api-access-dwsps\") pod \"maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn\" (UID: \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:09:52.477333 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.477323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-maas-api-tls\") pod \"maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn\" (UID: \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:09:52.477586 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.477357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-openshift-service-ca\") pod \"maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn\" (UID: \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:09:52.478170 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.478140    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-openshift-service-ca\") pod \"maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn\" (UID: \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:09:52.480314 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.480286    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-maas-api-tls\") pod \"maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn\" (UID: \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:09:52.490686 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.490659    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwsps\" (UniqueName: \"kubernetes.io/projected/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-kube-api-access-dwsps\") pod \"maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn\" (UID: \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:09:52.527752 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.527730    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:09:52.666717 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.666637    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"]
Jul 24 19:09:52.669313 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:09:52.669282    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9dd95a7_8c44_49c7_b58c_eddc8b0df651.slice/crio-3f97fbb7095dc63c76f21e7096130d368cad2eee6d044cf9388378072cd675ff WatchSource:0}: Error finding container 3f97fbb7095dc63c76f21e7096130d368cad2eee6d044cf9388378072cd675ff: Status 404 returned error can't find the container with id 3f97fbb7095dc63c76f21e7096130d368cad2eee6d044cf9388378072cd675ff
Jul 24 19:09:52.671026 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.671004    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 19:09:52.952326 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.952294    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"]
Jul 24 19:09:52.956434 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.956417    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"
Jul 24 19:09:52.959168 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.959145    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-d198c699-dockercfg-wxt58\""
Jul 24 19:09:52.959278 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.959182    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-d198c699\""
Jul 24 19:09:52.967758 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:52.967731    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"]
Jul 24 19:09:53.082855 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:53.082818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc30768e-9450-46a3-962f-564ed2bcd90e-plugins-config-volume\") pod \"payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr\" (UID: \"dc30768e-9450-46a3-962f-564ed2bcd90e\") " pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"
Jul 24 19:09:53.083037 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:53.082890    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkrcg\" (UniqueName: \"kubernetes.io/projected/dc30768e-9450-46a3-962f-564ed2bcd90e-kube-api-access-vkrcg\") pod \"payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr\" (UID: \"dc30768e-9450-46a3-962f-564ed2bcd90e\") " pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"
Jul 24 19:09:53.183813 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:53.183773    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkrcg\" (UniqueName: \"kubernetes.io/projected/dc30768e-9450-46a3-962f-564ed2bcd90e-kube-api-access-vkrcg\") pod \"payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr\" (UID: \"dc30768e-9450-46a3-962f-564ed2bcd90e\") " pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"
Jul 24 19:09:53.183992 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:53.183841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc30768e-9450-46a3-962f-564ed2bcd90e-plugins-config-volume\") pod \"payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr\" (UID: \"dc30768e-9450-46a3-962f-564ed2bcd90e\") " pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"
Jul 24 19:09:53.184546 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:53.184504    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc30768e-9450-46a3-962f-564ed2bcd90e-plugins-config-volume\") pod \"payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr\" (UID: \"dc30768e-9450-46a3-962f-564ed2bcd90e\") " pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"
Jul 24 19:09:53.192315 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:53.192297    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkrcg\" (UniqueName: \"kubernetes.io/projected/dc30768e-9450-46a3-962f-564ed2bcd90e-kube-api-access-vkrcg\") pod \"payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr\" (UID: \"dc30768e-9450-46a3-962f-564ed2bcd90e\") " pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"
Jul 24 19:09:53.267195 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:53.267102    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"
Jul 24 19:09:53.306139 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:53.306096    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn" event={"ID":"d9dd95a7-8c44-49c7-b58c-eddc8b0df651","Type":"ContainerStarted","Data":"3f97fbb7095dc63c76f21e7096130d368cad2eee6d044cf9388378072cd675ff"}
Jul 24 19:09:53.412814 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:53.412786    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"]
Jul 24 19:09:53.415329 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:09:53.415296    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc30768e_9450_46a3_962f_564ed2bcd90e.slice/crio-8a7db10fd6403e2ef6b4ef96ee92ded444bd37c616956d62a8f5115616510138 WatchSource:0}: Error finding container 8a7db10fd6403e2ef6b4ef96ee92ded444bd37c616956d62a8f5115616510138: Status 404 returned error can't find the container with id 8a7db10fd6403e2ef6b4ef96ee92ded444bd37c616956d62a8f5115616510138
Jul 24 19:09:54.315025 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:54.314936    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr" event={"ID":"dc30768e-9450-46a3-962f-564ed2bcd90e","Type":"ContainerStarted","Data":"8a7db10fd6403e2ef6b4ef96ee92ded444bd37c616956d62a8f5115616510138"}
Jul 24 19:09:56.328123 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:56.328086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr" event={"ID":"dc30768e-9450-46a3-962f-564ed2bcd90e","Type":"ContainerStarted","Data":"6251b9f16be134473a4a061b909d677b763260da0018385364d30bf627427006"}
Jul 24 19:09:56.328614 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:56.328151    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"
Jul 24 19:09:56.329788 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:56.329762    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn" event={"ID":"d9dd95a7-8c44-49c7-b58c-eddc8b0df651","Type":"ContainerStarted","Data":"c6de9b16274b936116b90a5f0d09b1bb6414ced022db4f8a4a0ce69334454282"}
Jul 24 19:09:56.330078 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:56.329897    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:09:56.346142 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:56.346097    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr" podStartSLOduration=2.233599506 podStartE2EDuration="4.346083691s" totalImagesPullingTime="2.112484185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:09:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:09:56.343150671 +0000 UTC m=+1534.734102149" watchObservedRunningTime="2026-07-24 19:09:56.346083691 +0000 UTC m=+1534.737035212"
Jul 24 19:09:56.362181 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:56.362129    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn" podStartSLOduration=1.5055302560000001 podStartE2EDuration="4.362114317s" totalImagesPullingTime="2.856584061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:09:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:09:56.360357429 +0000 UTC m=+1534.751308917" watchObservedRunningTime="2026-07-24 19:09:56.362114317 +0000 UTC m=+1534.753065793"
Jul 24 19:09:57.550117 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:09:57.550073    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:10:02.339800 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:02.339770    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:10:07.336822 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:07.336789    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"
Jul 24 19:10:28.112628 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.112593    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"]
Jul 24 19:10:28.113207 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.112850    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn" podUID="d9dd95a7-8c44-49c7-b58c-eddc8b0df651" containerName="maas-api" containerID="cri-o://c6de9b16274b936116b90a5f0d09b1bb6414ced022db4f8a4a0ce69334454282" gracePeriod=30
Jul 24 19:10:28.168481 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.168446    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"]
Jul 24 19:10:28.168715 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.168671    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr" podUID="dc30768e-9450-46a3-962f-564ed2bcd90e" containerName="payload-processing" containerID="cri-o://6251b9f16be134473a4a061b909d677b763260da0018385364d30bf627427006" gracePeriod=30
Jul 24 19:10:28.413218 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.413190    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:10:28.442205 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.442179    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"
Jul 24 19:10:28.486433 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.486380    2563 generic.go:356] "Generic (PLEG): container finished" podID="d9dd95a7-8c44-49c7-b58c-eddc8b0df651" containerID="c6de9b16274b936116b90a5f0d09b1bb6414ced022db4f8a4a0ce69334454282" exitCode=0
Jul 24 19:10:28.486433 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.486436    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"
Jul 24 19:10:28.486672 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.486467    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn" event={"ID":"d9dd95a7-8c44-49c7-b58c-eddc8b0df651","Type":"ContainerDied","Data":"c6de9b16274b936116b90a5f0d09b1bb6414ced022db4f8a4a0ce69334454282"}
Jul 24 19:10:28.486672 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.486532    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn" event={"ID":"d9dd95a7-8c44-49c7-b58c-eddc8b0df651","Type":"ContainerDied","Data":"3f97fbb7095dc63c76f21e7096130d368cad2eee6d044cf9388378072cd675ff"}
Jul 24 19:10:28.486672 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.486563    2563 scope.go:117] "RemoveContainer" containerID="c6de9b16274b936116b90a5f0d09b1bb6414ced022db4f8a4a0ce69334454282"
Jul 24 19:10:28.487811 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.487791    2563 generic.go:356] "Generic (PLEG): container finished" podID="dc30768e-9450-46a3-962f-564ed2bcd90e" containerID="6251b9f16be134473a4a061b909d677b763260da0018385364d30bf627427006" exitCode=0
Jul 24 19:10:28.487874 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.487833    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"
Jul 24 19:10:28.487874 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.487862    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr" event={"ID":"dc30768e-9450-46a3-962f-564ed2bcd90e","Type":"ContainerDied","Data":"6251b9f16be134473a4a061b909d677b763260da0018385364d30bf627427006"}
Jul 24 19:10:28.487941 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.487898    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr" event={"ID":"dc30768e-9450-46a3-962f-564ed2bcd90e","Type":"ContainerDied","Data":"8a7db10fd6403e2ef6b4ef96ee92ded444bd37c616956d62a8f5115616510138"}
Jul 24 19:10:28.497621 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.497601    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwsps\" (UniqueName: \"kubernetes.io/projected/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-kube-api-access-dwsps\") pod \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\" (UID: \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\") "
Jul 24 19:10:28.497703 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.497621    2563 scope.go:117] "RemoveContainer" containerID="c6de9b16274b936116b90a5f0d09b1bb6414ced022db4f8a4a0ce69334454282"
Jul 24 19:10:28.497760 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.497699    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-maas-api-tls\") pod \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\" (UID: \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\") "
Jul 24 19:10:28.497842 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.497825    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-openshift-service-ca\") pod \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\" (UID: \"d9dd95a7-8c44-49c7-b58c-eddc8b0df651\") "
Jul 24 19:10:28.498146 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.498122    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9dd95a7-8c44-49c7-b58c-eddc8b0df651" (UID: "d9dd95a7-8c44-49c7-b58c-eddc8b0df651"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:10:28.498768 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:10:28.498730    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6de9b16274b936116b90a5f0d09b1bb6414ced022db4f8a4a0ce69334454282\": container with ID starting with c6de9b16274b936116b90a5f0d09b1bb6414ced022db4f8a4a0ce69334454282 not found: ID does not exist" containerID="c6de9b16274b936116b90a5f0d09b1bb6414ced022db4f8a4a0ce69334454282"
Jul 24 19:10:28.498876 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.498779    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6de9b16274b936116b90a5f0d09b1bb6414ced022db4f8a4a0ce69334454282"} err="failed to get container status \"c6de9b16274b936116b90a5f0d09b1bb6414ced022db4f8a4a0ce69334454282\": rpc error: code = NotFound desc = could not find container \"c6de9b16274b936116b90a5f0d09b1bb6414ced022db4f8a4a0ce69334454282\": container with ID starting with c6de9b16274b936116b90a5f0d09b1bb6414ced022db4f8a4a0ce69334454282 not found: ID does not exist"
Jul 24 19:10:28.498876 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.498806    2563 scope.go:117] "RemoveContainer" containerID="6251b9f16be134473a4a061b909d677b763260da0018385364d30bf627427006"
Jul 24 19:10:28.500181 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.500160    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9dd95a7-8c44-49c7-b58c-eddc8b0df651" (UID: "d9dd95a7-8c44-49c7-b58c-eddc8b0df651"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:10:28.500277 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.500259    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-kube-api-access-dwsps" (OuterVolumeSpecName: "kube-api-access-dwsps") pod "d9dd95a7-8c44-49c7-b58c-eddc8b0df651" (UID: "d9dd95a7-8c44-49c7-b58c-eddc8b0df651"). InnerVolumeSpecName "kube-api-access-dwsps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:10:28.507069 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.507053    2563 scope.go:117] "RemoveContainer" containerID="6251b9f16be134473a4a061b909d677b763260da0018385364d30bf627427006"
Jul 24 19:10:28.507295 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:10:28.507276    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6251b9f16be134473a4a061b909d677b763260da0018385364d30bf627427006\": container with ID starting with 6251b9f16be134473a4a061b909d677b763260da0018385364d30bf627427006 not found: ID does not exist" containerID="6251b9f16be134473a4a061b909d677b763260da0018385364d30bf627427006"
Jul 24 19:10:28.507332 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.507307    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6251b9f16be134473a4a061b909d677b763260da0018385364d30bf627427006"} err="failed to get container status \"6251b9f16be134473a4a061b909d677b763260da0018385364d30bf627427006\": rpc error: code = NotFound desc = could not find container \"6251b9f16be134473a4a061b909d677b763260da0018385364d30bf627427006\": container with ID starting with 6251b9f16be134473a4a061b909d677b763260da0018385364d30bf627427006 not found: ID does not exist"
Jul 24 19:10:28.599075 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.599042    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkrcg\" (UniqueName: \"kubernetes.io/projected/dc30768e-9450-46a3-962f-564ed2bcd90e-kube-api-access-vkrcg\") pod \"dc30768e-9450-46a3-962f-564ed2bcd90e\" (UID: \"dc30768e-9450-46a3-962f-564ed2bcd90e\") "
Jul 24 19:10:28.599247 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.599152    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc30768e-9450-46a3-962f-564ed2bcd90e-plugins-config-volume\") pod \"dc30768e-9450-46a3-962f-564ed2bcd90e\" (UID: \"dc30768e-9450-46a3-962f-564ed2bcd90e\") "
Jul 24 19:10:28.599361 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.599346    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:10:28.599444 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.599369    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwsps\" (UniqueName: \"kubernetes.io/projected/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-kube-api-access-dwsps\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:10:28.599444 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.599384    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9dd95a7-8c44-49c7-b58c-eddc8b0df651-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:10:28.599529 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.599508    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc30768e-9450-46a3-962f-564ed2bcd90e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc30768e-9450-46a3-962f-564ed2bcd90e" (UID: "dc30768e-9450-46a3-962f-564ed2bcd90e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:10:28.601329 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.601308    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc30768e-9450-46a3-962f-564ed2bcd90e-kube-api-access-vkrcg" (OuterVolumeSpecName: "kube-api-access-vkrcg") pod "dc30768e-9450-46a3-962f-564ed2bcd90e" (UID: "dc30768e-9450-46a3-962f-564ed2bcd90e"). InnerVolumeSpecName "kube-api-access-vkrcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:10:28.699960 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.699875    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc30768e-9450-46a3-962f-564ed2bcd90e-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:10:28.699960 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.699905    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkrcg\" (UniqueName: \"kubernetes.io/projected/dc30768e-9450-46a3-962f-564ed2bcd90e-kube-api-access-vkrcg\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:10:28.814258 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.814234    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"]
Jul 24 19:10:28.818258 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.818235    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d198c699-84fdf68dbb-mwsfr"]
Jul 24 19:10:28.828170 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.828141    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"]
Jul 24 19:10:28.832279 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:28.832256    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d198c699-786f47bfc4-bgxfn"]
Jul 24 19:10:30.204428 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:30.204374    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9dd95a7-8c44-49c7-b58c-eddc8b0df651" path="/var/lib/kubelet/pods/d9dd95a7-8c44-49c7-b58c-eddc8b0df651/volumes"
Jul 24 19:10:30.204797 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:30.204756    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc30768e-9450-46a3-962f-564ed2bcd90e" path="/var/lib/kubelet/pods/dc30768e-9450-46a3-962f-564ed2bcd90e/volumes"
Jul 24 19:10:31.049895 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:31.049856    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:10:33.716757 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.716723    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"]
Jul 24 19:10:33.717141 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.717010    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9dd95a7-8c44-49c7-b58c-eddc8b0df651" containerName="maas-api"
Jul 24 19:10:33.717141 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.717020    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9dd95a7-8c44-49c7-b58c-eddc8b0df651" containerName="maas-api"
Jul 24 19:10:33.717141 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.717036    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc30768e-9450-46a3-962f-564ed2bcd90e" containerName="payload-processing"
Jul 24 19:10:33.717141 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.717041    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc30768e-9450-46a3-962f-564ed2bcd90e" containerName="payload-processing"
Jul 24 19:10:33.717141 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.717094    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc30768e-9450-46a3-962f-564ed2bcd90e" containerName="payload-processing"
Jul 24 19:10:33.717141 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.717102    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9dd95a7-8c44-49c7-b58c-eddc8b0df651" containerName="maas-api"
Jul 24 19:10:33.719921 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.719897    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.722403 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.722373    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-e0dcb0ab-gw-openshift-default-dockercfg-m49bv\""
Jul 24 19:10:33.730924 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.730901    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"]
Jul 24 19:10:33.847246 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.847213    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-podinfo\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.847426 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.847260    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-workload-certs\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.847426 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.847333    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf8mn\" (UniqueName: \"kubernetes.io/projected/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-kube-api-access-bf8mn\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.847426 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.847420    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-credential-socket\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.847540 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.847440    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-token\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.847540 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.847459    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-workload-socket\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.847540 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.847476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-data\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.847540 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.847497    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istiod-ca-cert\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.847656 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.847551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-envoy\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.948985 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.948931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-envoy\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.949167 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.949006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-podinfo\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.949167 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.949033    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-workload-certs\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.949167 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.949066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf8mn\" (UniqueName: \"kubernetes.io/projected/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-kube-api-access-bf8mn\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.949167 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.949128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-credential-socket\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.949167 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.949149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-token\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.949433 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.949173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-workload-socket\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.949433 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.949200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-data\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.949433 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.949233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istiod-ca-cert\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.949588 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.949514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-workload-certs\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.949646 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.949584    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-workload-socket\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.949693 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.949634    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-credential-socket\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.949863 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.949785    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-data\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.950099 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.950067    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istiod-ca-cert\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.951608 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.951583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-envoy\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.951717 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.951701    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-podinfo\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.957158 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.957119    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-token\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:33.957298 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:33.957284    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf8mn\" (UniqueName: \"kubernetes.io/projected/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-kube-api-access-bf8mn\") pod \"e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") " pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:34.030889 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:34.030798    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:34.168369 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:34.168343    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"]
Jul 24 19:10:34.170825 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:10:34.170783    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf14dbbe_f439_49f6_bcf1_5bd35f9cc385.slice/crio-ee520cffb1eead78c3b2eab9c89a0d2c6758560c64f9c5e7d5f67898380cc41a WatchSource:0}: Error finding container ee520cffb1eead78c3b2eab9c89a0d2c6758560c64f9c5e7d5f67898380cc41a: Status 404 returned error can't find the container with id ee520cffb1eead78c3b2eab9c89a0d2c6758560c64f9c5e7d5f67898380cc41a
Jul 24 19:10:34.520708 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:34.520670    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs" event={"ID":"df14dbbe-f439-49f6-bcf1-5bd35f9cc385","Type":"ContainerStarted","Data":"ee520cffb1eead78c3b2eab9c89a0d2c6758560c64f9c5e7d5f67898380cc41a"}
Jul 24 19:10:36.649562 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:36.649524    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:10:36.649822 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:36.649599    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:10:36.649822 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:36.649631    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:10:37.540339 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:37.540296    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs" event={"ID":"df14dbbe-f439-49f6-bcf1-5bd35f9cc385","Type":"ContainerStarted","Data":"75c691731c274c8f68f57da2893fec2e23e1a4f1ae0f4ff3aa4a0a5dc94259db"}
Jul 24 19:10:37.575184 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:37.575135    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs" podStartSLOduration=2.098876554 podStartE2EDuration="4.575119565s" totalImagesPullingTime="2.476243011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:10:37.573664424 +0000 UTC m=+1575.964615903" watchObservedRunningTime="2026-07-24 19:10:37.575119565 +0000 UTC m=+1575.966071041"
Jul 24 19:10:38.031354 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:38.031313    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:38.035963 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:38.035941    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:38.545503 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:38.545477    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:38.546567 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:38.546548    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:39.123306 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.123274    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"]
Jul 24 19:10:39.125674 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.125656    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:10:39.127912 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.127886    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e0dcb0ab-serving-cert\""
Jul 24 19:10:39.128030 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.128013    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 19:10:39.128130 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.128110    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 19:10:39.128805 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.128784    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cvjqn\""
Jul 24 19:10:39.137247 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.137226    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"]
Jul 24 19:10:39.294726 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.294695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-maas-api-tls\") pod \"maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k\" (UID: \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:10:39.294726 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.294730    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6l8f\" (UniqueName: \"kubernetes.io/projected/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-kube-api-access-t6l8f\") pod \"maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k\" (UID: \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:10:39.294943 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.294847    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-openshift-service-ca\") pod \"maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k\" (UID: \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:10:39.395465 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.395358    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-openshift-service-ca\") pod \"maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k\" (UID: \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:10:39.395465 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.395443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-maas-api-tls\") pod \"maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k\" (UID: \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:10:39.395667 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.395475    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6l8f\" (UniqueName: \"kubernetes.io/projected/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-kube-api-access-t6l8f\") pod \"maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k\" (UID: \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:10:39.396034 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.396014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-openshift-service-ca\") pod \"maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k\" (UID: \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:10:39.398044 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.398022    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-maas-api-tls\") pod \"maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k\" (UID: \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:10:39.405019 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.404996    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6l8f\" (UniqueName: \"kubernetes.io/projected/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-kube-api-access-t6l8f\") pod \"maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k\" (UID: \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:10:39.435749 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.435720    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:10:39.567685 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.567654    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"]
Jul 24 19:10:39.571621 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:10:39.571593    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dbc3532_4534_4a06_b5b9_372d8e6a4b24.slice/crio-53c1612c8f5d6e4e01b82a1948d696ac7100bd2056f2aeebfb663de24a6fb531 WatchSource:0}: Error finding container 53c1612c8f5d6e4e01b82a1948d696ac7100bd2056f2aeebfb663de24a6fb531: Status 404 returned error can't find the container with id 53c1612c8f5d6e4e01b82a1948d696ac7100bd2056f2aeebfb663de24a6fb531
Jul 24 19:10:39.783221 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.783145    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"]
Jul 24 19:10:39.786218 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.786202    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"
Jul 24 19:10:39.788610 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.788587    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-e0dcb0ab-dockercfg-jwdh7\""
Jul 24 19:10:39.788713 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.788633    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-e0dcb0ab\""
Jul 24 19:10:39.797010 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.796988    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"]
Jul 24 19:10:39.804730 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.804711    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"]
Jul 24 19:10:39.806951 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.806935    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"
Jul 24 19:10:39.822027 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.822004    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"]
Jul 24 19:10:39.898487 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.898460    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abad3f19-80df-4579-9ae2-78f4a92a96af-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx\" (UID: \"abad3f19-80df-4579-9ae2-78f4a92a96af\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"
Jul 24 19:10:39.898631 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.898520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qxsz\" (UniqueName: \"kubernetes.io/projected/abad3f19-80df-4579-9ae2-78f4a92a96af-kube-api-access-2qxsz\") pod \"payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx\" (UID: \"abad3f19-80df-4579-9ae2-78f4a92a96af\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"
Jul 24 19:10:39.999706 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.999665    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh62x\" (UniqueName: \"kubernetes.io/projected/5b7dbe61-e89f-4c72-88fc-40ad5f46be80-kube-api-access-sh62x\") pod \"payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c\" (UID: \"5b7dbe61-e89f-4c72-88fc-40ad5f46be80\") " pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"
Jul 24 19:10:39.999878 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.999714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abad3f19-80df-4579-9ae2-78f4a92a96af-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx\" (UID: \"abad3f19-80df-4579-9ae2-78f4a92a96af\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"
Jul 24 19:10:39.999878 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.999777    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7dbe61-e89f-4c72-88fc-40ad5f46be80-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c\" (UID: \"5b7dbe61-e89f-4c72-88fc-40ad5f46be80\") " pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"
Jul 24 19:10:39.999878 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:39.999868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qxsz\" (UniqueName: \"kubernetes.io/projected/abad3f19-80df-4579-9ae2-78f4a92a96af-kube-api-access-2qxsz\") pod \"payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx\" (UID: \"abad3f19-80df-4579-9ae2-78f4a92a96af\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"
Jul 24 19:10:40.000446 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.000426    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abad3f19-80df-4579-9ae2-78f4a92a96af-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx\" (UID: \"abad3f19-80df-4579-9ae2-78f4a92a96af\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"
Jul 24 19:10:40.008561 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.008527    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qxsz\" (UniqueName: \"kubernetes.io/projected/abad3f19-80df-4579-9ae2-78f4a92a96af-kube-api-access-2qxsz\") pod \"payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx\" (UID: \"abad3f19-80df-4579-9ae2-78f4a92a96af\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"
Jul 24 19:10:40.097300 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.097263    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"
Jul 24 19:10:40.100732 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.100709    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh62x\" (UniqueName: \"kubernetes.io/projected/5b7dbe61-e89f-4c72-88fc-40ad5f46be80-kube-api-access-sh62x\") pod \"payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c\" (UID: \"5b7dbe61-e89f-4c72-88fc-40ad5f46be80\") " pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"
Jul 24 19:10:40.100775 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.100751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7dbe61-e89f-4c72-88fc-40ad5f46be80-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c\" (UID: \"5b7dbe61-e89f-4c72-88fc-40ad5f46be80\") " pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"
Jul 24 19:10:40.101754 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.101733    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7dbe61-e89f-4c72-88fc-40ad5f46be80-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c\" (UID: \"5b7dbe61-e89f-4c72-88fc-40ad5f46be80\") " pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"
Jul 24 19:10:40.109567 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.109538    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh62x\" (UniqueName: \"kubernetes.io/projected/5b7dbe61-e89f-4c72-88fc-40ad5f46be80-kube-api-access-sh62x\") pod \"payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c\" (UID: \"5b7dbe61-e89f-4c72-88fc-40ad5f46be80\") " pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"
Jul 24 19:10:40.117872 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.117849    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"
Jul 24 19:10:40.245382 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.245350    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"]
Jul 24 19:10:40.247948 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:10:40.247916    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabad3f19_80df_4579_9ae2_78f4a92a96af.slice/crio-544d1dfb11861406d8c7426e9c0f2b2b8e675d08d303939a544d143816774a1d WatchSource:0}: Error finding container 544d1dfb11861406d8c7426e9c0f2b2b8e675d08d303939a544d143816774a1d: Status 404 returned error can't find the container with id 544d1dfb11861406d8c7426e9c0f2b2b8e675d08d303939a544d143816774a1d
Jul 24 19:10:40.278265 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.278235    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"]
Jul 24 19:10:40.279576 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:10:40.279543    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7dbe61_e89f_4c72_88fc_40ad5f46be80.slice/crio-8b4074bdeb9a9045f852c3c902f148e04ee8609918e9fe5e34858d90066131ba WatchSource:0}: Error finding container 8b4074bdeb9a9045f852c3c902f148e04ee8609918e9fe5e34858d90066131ba: Status 404 returned error can't find the container with id 8b4074bdeb9a9045f852c3c902f148e04ee8609918e9fe5e34858d90066131ba
Jul 24 19:10:40.558660 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.558624    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c" event={"ID":"5b7dbe61-e89f-4c72-88fc-40ad5f46be80","Type":"ContainerStarted","Data":"28914a48d89e6273806f2ea52114e33ce3e9d3227b2fe3ee7ecbd5c61eb71b49"}
Jul 24 19:10:40.558892 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.558688    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c" event={"ID":"5b7dbe61-e89f-4c72-88fc-40ad5f46be80","Type":"ContainerStarted","Data":"8b4074bdeb9a9045f852c3c902f148e04ee8609918e9fe5e34858d90066131ba"}
Jul 24 19:10:40.558892 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.558736    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"
Jul 24 19:10:40.560262 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.560234    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k" event={"ID":"4dbc3532-4534-4a06-b5b9-372d8e6a4b24","Type":"ContainerStarted","Data":"f3ea2c958a37d2acac49b8898acb7734a7c1c74d7b1a71beb7db7b6ed4c0c3db"}
Jul 24 19:10:40.560380 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.560278    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k" event={"ID":"4dbc3532-4534-4a06-b5b9-372d8e6a4b24","Type":"ContainerStarted","Data":"53c1612c8f5d6e4e01b82a1948d696ac7100bd2056f2aeebfb663de24a6fb531"}
Jul 24 19:10:40.560380 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.560297    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:10:40.561776 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.561727    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx" event={"ID":"abad3f19-80df-4579-9ae2-78f4a92a96af","Type":"ContainerStarted","Data":"d59326ca31a4f729b07287e187a0d0d80fa3a9e3c3d132ab9bcaf54dc1e2f979"}
Jul 24 19:10:40.561776 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.561771    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx" event={"ID":"abad3f19-80df-4579-9ae2-78f4a92a96af","Type":"ContainerStarted","Data":"544d1dfb11861406d8c7426e9c0f2b2b8e675d08d303939a544d143816774a1d"}
Jul 24 19:10:40.561953 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.561928    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"
Jul 24 19:10:40.576976 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.576937    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c" podStartSLOduration=1.576924414 podStartE2EDuration="1.576924414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:10:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:10:40.575425942 +0000 UTC m=+1578.966377416" watchObservedRunningTime="2026-07-24 19:10:40.576924414 +0000 UTC m=+1578.967875891"
Jul 24 19:10:40.592341 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.592302    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k" podStartSLOduration=1.301281571 podStartE2EDuration="1.592289757s" totalImagesPullingTime="291.008186ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:10:40.590817422 +0000 UTC m=+1578.981768902" watchObservedRunningTime="2026-07-24 19:10:40.592289757 +0000 UTC m=+1578.983241234"
Jul 24 19:10:40.606520 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:40.606482    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx" podStartSLOduration=1.606474 podStartE2EDuration="1.606474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:10:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:10:40.604788642 +0000 UTC m=+1578.995740141" watchObservedRunningTime="2026-07-24 19:10:40.606474 +0000 UTC m=+1578.997425476"
Jul 24 19:10:43.354223 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:43.354187    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:10:46.572695 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:46.572662    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:10:51.569419 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:51.569374    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"
Jul 24 19:10:51.569920 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:51.569619    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"
Jul 24 19:10:51.618779 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:51.618736    2563 generic.go:356] "Generic (PLEG): container finished" podID="df14dbbe-f439-49f6-bcf1-5bd35f9cc385" containerID="75c691731c274c8f68f57da2893fec2e23e1a4f1ae0f4ff3aa4a0a5dc94259db" exitCode=0
Jul 24 19:10:51.618941 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:51.618815    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs" event={"ID":"df14dbbe-f439-49f6-bcf1-5bd35f9cc385","Type":"ContainerDied","Data":"75c691731c274c8f68f57da2893fec2e23e1a4f1ae0f4ff3aa4a0a5dc94259db"}
Jul 24 19:10:51.619323 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:51.619301    2563 scope.go:117] "RemoveContainer" containerID="75c691731c274c8f68f57da2893fec2e23e1a4f1ae0f4ff3aa4a0a5dc94259db"
Jul 24 19:10:51.621470 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:51.621428    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:10:51.621565 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:51.621530    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:10:51.621605 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:51.621582    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:10:52.630347 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:52.630299    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs" event={"ID":"df14dbbe-f439-49f6-bcf1-5bd35f9cc385","Type":"ContainerStarted","Data":"ed5acb59806ffda3c0160de1860492fbc1b695d31d22450bd8d43d2261281a95"}
Jul 24 19:10:53.031882 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:53.031775    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:53.033000 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:53.032961    2563 patch_prober.go:28] interesting pod/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 24 19:10:53.033118 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:53.033043    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs" podUID="df14dbbe-f439-49f6-bcf1-5bd35f9cc385" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 24 19:10:53.545976 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:53.545944    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:54.031560 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:54.031516    2563 patch_prober.go:28] interesting pod/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 24 19:10:54.032072 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:54.031604    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs" podUID="df14dbbe-f439-49f6-bcf1-5bd35f9cc385" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 24 19:10:55.031518 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:55.031475    2563 patch_prober.go:28] interesting pod/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 24 19:10:55.031724 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:55.031560    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs" podUID="df14dbbe-f439-49f6-bcf1-5bd35f9cc385" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 24 19:10:56.031870 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:56.031824    2563 patch_prober.go:28] interesting pod/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 24 19:10:56.032370 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:56.031900    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs" podUID="df14dbbe-f439-49f6-bcf1-5bd35f9cc385" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 24 19:10:57.035707 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:57.035679    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:10:57.036639 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:10:57.036621    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:11:14.568678 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:14.568594    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"]
Jul 24 19:11:14.569220 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:14.568825    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k" podUID="4dbc3532-4534-4a06-b5b9-372d8e6a4b24" containerName="maas-api" containerID="cri-o://f3ea2c958a37d2acac49b8898acb7734a7c1c74d7b1a71beb7db7b6ed4c0c3db" gracePeriod=30
Jul 24 19:11:14.628815 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:14.628781    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"]
Jul 24 19:11:14.629022 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:14.628996    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c" podUID="5b7dbe61-e89f-4c72-88fc-40ad5f46be80" containerName="payload-processing" containerID="cri-o://28914a48d89e6273806f2ea52114e33ce3e9d3227b2fe3ee7ecbd5c61eb71b49" gracePeriod=30
Jul 24 19:11:14.634778 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:14.634738    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"]
Jul 24 19:11:14.635039 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:14.634992    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx" podUID="abad3f19-80df-4579-9ae2-78f4a92a96af" containerName="payload-pre-processing" containerID="cri-o://d59326ca31a4f729b07287e187a0d0d80fa3a9e3c3d132ab9bcaf54dc1e2f979" gracePeriod=30
Jul 24 19:11:14.748536 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:14.748503    2563 generic.go:356] "Generic (PLEG): container finished" podID="5b7dbe61-e89f-4c72-88fc-40ad5f46be80" containerID="28914a48d89e6273806f2ea52114e33ce3e9d3227b2fe3ee7ecbd5c61eb71b49" exitCode=0
Jul 24 19:11:14.748651 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:14.748602    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c" event={"ID":"5b7dbe61-e89f-4c72-88fc-40ad5f46be80","Type":"ContainerDied","Data":"28914a48d89e6273806f2ea52114e33ce3e9d3227b2fe3ee7ecbd5c61eb71b49"}
Jul 24 19:11:14.750577 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:14.750541    2563 generic.go:356] "Generic (PLEG): container finished" podID="4dbc3532-4534-4a06-b5b9-372d8e6a4b24" containerID="f3ea2c958a37d2acac49b8898acb7734a7c1c74d7b1a71beb7db7b6ed4c0c3db" exitCode=0
Jul 24 19:11:14.750671 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:14.750616    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k" event={"ID":"4dbc3532-4534-4a06-b5b9-372d8e6a4b24","Type":"ContainerDied","Data":"f3ea2c958a37d2acac49b8898acb7734a7c1c74d7b1a71beb7db7b6ed4c0c3db"}
Jul 24 19:11:14.897437 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:14.897409    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:11:14.933576 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:14.933552    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"
Jul 24 19:11:14.936553 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:14.936534    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"
Jul 24 19:11:15.014007 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.013969    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-maas-api-tls\") pod \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\" (UID: \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\") "
Jul 24 19:11:15.014198 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.014014    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6l8f\" (UniqueName: \"kubernetes.io/projected/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-kube-api-access-t6l8f\") pod \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\" (UID: \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\") "
Jul 24 19:11:15.014198 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.014040    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-openshift-service-ca\") pod \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\" (UID: \"4dbc3532-4534-4a06-b5b9-372d8e6a4b24\") "
Jul 24 19:11:15.014198 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.014062    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sh62x\" (UniqueName: \"kubernetes.io/projected/5b7dbe61-e89f-4c72-88fc-40ad5f46be80-kube-api-access-sh62x\") pod \"5b7dbe61-e89f-4c72-88fc-40ad5f46be80\" (UID: \"5b7dbe61-e89f-4c72-88fc-40ad5f46be80\") "
Jul 24 19:11:15.014198 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.014113    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7dbe61-e89f-4c72-88fc-40ad5f46be80-plugins-config-volume\") pod \"5b7dbe61-e89f-4c72-88fc-40ad5f46be80\" (UID: \"5b7dbe61-e89f-4c72-88fc-40ad5f46be80\") "
Jul 24 19:11:15.014198 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.014182    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abad3f19-80df-4579-9ae2-78f4a92a96af-plugins-config-volume\") pod \"abad3f19-80df-4579-9ae2-78f4a92a96af\" (UID: \"abad3f19-80df-4579-9ae2-78f4a92a96af\") "
Jul 24 19:11:15.014479 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.014215    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qxsz\" (UniqueName: \"kubernetes.io/projected/abad3f19-80df-4579-9ae2-78f4a92a96af-kube-api-access-2qxsz\") pod \"abad3f19-80df-4579-9ae2-78f4a92a96af\" (UID: \"abad3f19-80df-4579-9ae2-78f4a92a96af\") "
Jul 24 19:11:15.014540 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.014499    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4dbc3532-4534-4a06-b5b9-372d8e6a4b24" (UID: "4dbc3532-4534-4a06-b5b9-372d8e6a4b24"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:11:15.014598 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.014574    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b7dbe61-e89f-4c72-88fc-40ad5f46be80-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5b7dbe61-e89f-4c72-88fc-40ad5f46be80" (UID: "5b7dbe61-e89f-4c72-88fc-40ad5f46be80"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:11:15.014656 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.014596    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abad3f19-80df-4579-9ae2-78f4a92a96af-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "abad3f19-80df-4579-9ae2-78f4a92a96af" (UID: "abad3f19-80df-4579-9ae2-78f4a92a96af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:11:15.016306 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.016281    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b7dbe61-e89f-4c72-88fc-40ad5f46be80-kube-api-access-sh62x" (OuterVolumeSpecName: "kube-api-access-sh62x") pod "5b7dbe61-e89f-4c72-88fc-40ad5f46be80" (UID: "5b7dbe61-e89f-4c72-88fc-40ad5f46be80"). InnerVolumeSpecName "kube-api-access-sh62x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:11:15.016449 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.016409    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abad3f19-80df-4579-9ae2-78f4a92a96af-kube-api-access-2qxsz" (OuterVolumeSpecName: "kube-api-access-2qxsz") pod "abad3f19-80df-4579-9ae2-78f4a92a96af" (UID: "abad3f19-80df-4579-9ae2-78f4a92a96af"). InnerVolumeSpecName "kube-api-access-2qxsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:11:15.016697 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.016681    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4dbc3532-4534-4a06-b5b9-372d8e6a4b24" (UID: "4dbc3532-4534-4a06-b5b9-372d8e6a4b24"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:11:15.016760 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.016742    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-kube-api-access-t6l8f" (OuterVolumeSpecName: "kube-api-access-t6l8f") pod "4dbc3532-4534-4a06-b5b9-372d8e6a4b24" (UID: "4dbc3532-4534-4a06-b5b9-372d8e6a4b24"). InnerVolumeSpecName "kube-api-access-t6l8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:11:15.115694 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.115608    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7dbe61-e89f-4c72-88fc-40ad5f46be80-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:15.115694 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.115636    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abad3f19-80df-4579-9ae2-78f4a92a96af-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:15.115694 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.115644    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qxsz\" (UniqueName: \"kubernetes.io/projected/abad3f19-80df-4579-9ae2-78f4a92a96af-kube-api-access-2qxsz\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:15.115694 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.115653    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:15.115694 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.115662    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6l8f\" (UniqueName: \"kubernetes.io/projected/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-kube-api-access-t6l8f\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:15.115694 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.115670    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbc3532-4534-4a06-b5b9-372d8e6a4b24-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:15.115694 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.115680    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sh62x\" (UniqueName: \"kubernetes.io/projected/5b7dbe61-e89f-4c72-88fc-40ad5f46be80-kube-api-access-sh62x\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:15.756644 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.756613    2563 generic.go:356] "Generic (PLEG): container finished" podID="abad3f19-80df-4579-9ae2-78f4a92a96af" containerID="d59326ca31a4f729b07287e187a0d0d80fa3a9e3c3d132ab9bcaf54dc1e2f979" exitCode=0
Jul 24 19:11:15.757063 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.756658    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"
Jul 24 19:11:15.757063 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.756701    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx" event={"ID":"abad3f19-80df-4579-9ae2-78f4a92a96af","Type":"ContainerDied","Data":"d59326ca31a4f729b07287e187a0d0d80fa3a9e3c3d132ab9bcaf54dc1e2f979"}
Jul 24 19:11:15.757063 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.756749    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx" event={"ID":"abad3f19-80df-4579-9ae2-78f4a92a96af","Type":"ContainerDied","Data":"544d1dfb11861406d8c7426e9c0f2b2b8e675d08d303939a544d143816774a1d"}
Jul 24 19:11:15.757063 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.756773    2563 scope.go:117] "RemoveContainer" containerID="d59326ca31a4f729b07287e187a0d0d80fa3a9e3c3d132ab9bcaf54dc1e2f979"
Jul 24 19:11:15.758119 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.758103    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"
Jul 24 19:11:15.758212 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.758117    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c" event={"ID":"5b7dbe61-e89f-4c72-88fc-40ad5f46be80","Type":"ContainerDied","Data":"8b4074bdeb9a9045f852c3c902f148e04ee8609918e9fe5e34858d90066131ba"}
Jul 24 19:11:15.759860 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.759828    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k" event={"ID":"4dbc3532-4534-4a06-b5b9-372d8e6a4b24","Type":"ContainerDied","Data":"53c1612c8f5d6e4e01b82a1948d696ac7100bd2056f2aeebfb663de24a6fb531"}
Jul 24 19:11:15.759970 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.759871    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"
Jul 24 19:11:15.767219 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.767198    2563 scope.go:117] "RemoveContainer" containerID="d59326ca31a4f729b07287e187a0d0d80fa3a9e3c3d132ab9bcaf54dc1e2f979"
Jul 24 19:11:15.767537 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:11:15.767507    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d59326ca31a4f729b07287e187a0d0d80fa3a9e3c3d132ab9bcaf54dc1e2f979\": container with ID starting with d59326ca31a4f729b07287e187a0d0d80fa3a9e3c3d132ab9bcaf54dc1e2f979 not found: ID does not exist" containerID="d59326ca31a4f729b07287e187a0d0d80fa3a9e3c3d132ab9bcaf54dc1e2f979"
Jul 24 19:11:15.767730 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.767548    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d59326ca31a4f729b07287e187a0d0d80fa3a9e3c3d132ab9bcaf54dc1e2f979"} err="failed to get container status \"d59326ca31a4f729b07287e187a0d0d80fa3a9e3c3d132ab9bcaf54dc1e2f979\": rpc error: code = NotFound desc = could not find container \"d59326ca31a4f729b07287e187a0d0d80fa3a9e3c3d132ab9bcaf54dc1e2f979\": container with ID starting with d59326ca31a4f729b07287e187a0d0d80fa3a9e3c3d132ab9bcaf54dc1e2f979 not found: ID does not exist"
Jul 24 19:11:15.767730 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.767572    2563 scope.go:117] "RemoveContainer" containerID="28914a48d89e6273806f2ea52114e33ce3e9d3227b2fe3ee7ecbd5c61eb71b49"
Jul 24 19:11:15.775692 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.775676    2563 scope.go:117] "RemoveContainer" containerID="f3ea2c958a37d2acac49b8898acb7734a7c1c74d7b1a71beb7db7b6ed4c0c3db"
Jul 24 19:11:15.782249 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.782223    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"]
Jul 24 19:11:15.785955 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.785932    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e0dcb0ab-6444c7d464-vf4wx"]
Jul 24 19:11:15.796581 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.796560    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"]
Jul 24 19:11:15.802275 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.802254    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e0dcb0ab-6dc5948997-dt25c"]
Jul 24 19:11:15.813860 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.813819    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"]
Jul 24 19:11:15.815533 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:15.815515    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0dcb0ab-85688bd467-nfv4k"]
Jul 24 19:11:16.203559 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:16.203523    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dbc3532-4534-4a06-b5b9-372d8e6a4b24" path="/var/lib/kubelet/pods/4dbc3532-4534-4a06-b5b9-372d8e6a4b24/volumes"
Jul 24 19:11:16.203894 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:16.203880    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b7dbe61-e89f-4c72-88fc-40ad5f46be80" path="/var/lib/kubelet/pods/5b7dbe61-e89f-4c72-88fc-40ad5f46be80/volumes"
Jul 24 19:11:16.204175 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:16.204164    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abad3f19-80df-4579-9ae2-78f4a92a96af" path="/var/lib/kubelet/pods/abad3f19-80df-4579-9ae2-78f4a92a96af/volumes"
Jul 24 19:11:17.847826 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:17.847791    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:11:19.251554 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:19.251521    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"]
Jul 24 19:11:19.251948 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:19.251771    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs" podUID="df14dbbe-f439-49f6-bcf1-5bd35f9cc385" containerName="istio-proxy" containerID="cri-o://ed5acb59806ffda3c0160de1860492fbc1b695d31d22450bd8d43d2261281a95" gracePeriod=30
Jul 24 19:11:20.038657 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.038627    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"]
Jul 24 19:11:20.038928 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.038916    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dbc3532-4534-4a06-b5b9-372d8e6a4b24" containerName="maas-api"
Jul 24 19:11:20.038973 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.038930    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dbc3532-4534-4a06-b5b9-372d8e6a4b24" containerName="maas-api"
Jul 24 19:11:20.038973 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.038940    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b7dbe61-e89f-4c72-88fc-40ad5f46be80" containerName="payload-processing"
Jul 24 19:11:20.038973 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.038945    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b7dbe61-e89f-4c72-88fc-40ad5f46be80" containerName="payload-processing"
Jul 24 19:11:20.038973 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.038964    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abad3f19-80df-4579-9ae2-78f4a92a96af" containerName="payload-pre-processing"
Jul 24 19:11:20.038973 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.038969    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="abad3f19-80df-4579-9ae2-78f4a92a96af" containerName="payload-pre-processing"
Jul 24 19:11:20.039121 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.039022    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="abad3f19-80df-4579-9ae2-78f4a92a96af" containerName="payload-pre-processing"
Jul 24 19:11:20.039121 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.039030    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b7dbe61-e89f-4c72-88fc-40ad5f46be80" containerName="payload-processing"
Jul 24 19:11:20.039121 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.039038    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dbc3532-4534-4a06-b5b9-372d8e6a4b24" containerName="maas-api"
Jul 24 19:11:20.041994 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.041979    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.044940 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.044915    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ef3668b1-openshift-default-dockercfg-hnmhh\""
Jul 24 19:11:20.054344 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.054324    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"]
Jul 24 19:11:20.162295 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.162258    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-envoy\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.162499 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.162302    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-workload-socket\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.162499 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.162346    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-token\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.162499 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.162369    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-credential-socket\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.162499 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.162409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-podinfo\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.162499 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.162464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bc6h\" (UniqueName: \"kubernetes.io/projected/05a69c7b-065e-4ce7-ac53-6e809d61a5af-kube-api-access-4bc6h\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.162668 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.162516    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-workload-certs\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.162668 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.162558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istiod-ca-cert\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.162668 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.162577    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-data\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.263580 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.263544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-workload-certs\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.263580 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.263590    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istiod-ca-cert\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.264103 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.263610    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-data\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.264103 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.263641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-envoy\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.264103 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.263662    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-workload-socket\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.264103 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.263680    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-token\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.264103 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.263698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-credential-socket\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.264103 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.263714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-podinfo\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.264103 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.263744    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bc6h\" (UniqueName: \"kubernetes.io/projected/05a69c7b-065e-4ce7-ac53-6e809d61a5af-kube-api-access-4bc6h\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.264103 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.264074    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-workload-certs\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.264412 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.264151    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-data\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.264412 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.264272    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-workload-socket\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.264412 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.264288    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-credential-socket\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.264582 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.264564    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istiod-ca-cert\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.266212 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.266133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-envoy\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.266434 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.266417    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-podinfo\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.272386 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.272349    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bc6h\" (UniqueName: \"kubernetes.io/projected/05a69c7b-065e-4ce7-ac53-6e809d61a5af-kube-api-access-4bc6h\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.272720 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.272700    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-token\") pod \"e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") " pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.353427 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.353380    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:20.503227 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.503199    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"]
Jul 24 19:11:20.504940 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:11:20.504915    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05a69c7b_065e_4ce7_ac53_6e809d61a5af.slice/crio-c01d227878c69689887a847a4e40577ff4a48db583a332e6a57a0fb90a91ae7c WatchSource:0}: Error finding container c01d227878c69689887a847a4e40577ff4a48db583a332e6a57a0fb90a91ae7c: Status 404 returned error can't find the container with id c01d227878c69689887a847a4e40577ff4a48db583a332e6a57a0fb90a91ae7c
Jul 24 19:11:20.506997 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.506956    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:11:20.507074 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.507041    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:11:20.507108 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.507083    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:11:20.792911 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.792817    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd" event={"ID":"05a69c7b-065e-4ce7-ac53-6e809d61a5af","Type":"ContainerStarted","Data":"2e24e6e65803940d00b9816ceb7d898b6a82b2e78c75701c1bb060ee909bff17"}
Jul 24 19:11:20.792911 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:20.792880    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd" event={"ID":"05a69c7b-065e-4ce7-ac53-6e809d61a5af","Type":"ContainerStarted","Data":"c01d227878c69689887a847a4e40577ff4a48db583a332e6a57a0fb90a91ae7c"}
Jul 24 19:11:21.353700 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:21.353666    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:21.358628 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:21.358605    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:21.377870 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:21.377817    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd" podStartSLOduration=1.377800811 podStartE2EDuration="1.377800811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:11:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:11:20.818742889 +0000 UTC m=+1619.209694368" watchObservedRunningTime="2026-07-24 19:11:21.377800811 +0000 UTC m=+1619.768752287"
Jul 24 19:11:21.798865 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:21.798782    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:21.799725 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:21.799706    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:22.256208 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.256183    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"]
Jul 24 19:11:22.259803 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.259785    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:22.262693 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.262668    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ef3668b1-serving-cert\""
Jul 24 19:11:22.262693 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.262690    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 19:11:22.263600 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.263580    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cvjqn\""
Jul 24 19:11:22.263675 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.263589    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 19:11:22.270022 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.269997    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"]
Jul 24 19:11:22.382620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.382585    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59d7f887-7f7e-490a-8295-0f4b23f76c0d-maas-api-tls\") pod \"maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb\" (UID: \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:22.383029 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.382628    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d7f887-7f7e-490a-8295-0f4b23f76c0d-openshift-service-ca\") pod \"maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb\" (UID: \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:22.383029 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.382707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd587\" (UniqueName: \"kubernetes.io/projected/59d7f887-7f7e-490a-8295-0f4b23f76c0d-kube-api-access-kd587\") pod \"maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb\" (UID: \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:22.483830 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.483735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd587\" (UniqueName: \"kubernetes.io/projected/59d7f887-7f7e-490a-8295-0f4b23f76c0d-kube-api-access-kd587\") pod \"maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb\" (UID: \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:22.483830 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.483805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59d7f887-7f7e-490a-8295-0f4b23f76c0d-maas-api-tls\") pod \"maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb\" (UID: \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:22.484059 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.483845    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d7f887-7f7e-490a-8295-0f4b23f76c0d-openshift-service-ca\") pod \"maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb\" (UID: \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:22.484059 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:11:22.483982    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-serving-cert: secret "maas-api-e2e-ait-ef3668b1-serving-cert" not found
Jul 24 19:11:22.484162 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:11:22.484060    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d7f887-7f7e-490a-8295-0f4b23f76c0d-maas-api-tls podName:59d7f887-7f7e-490a-8295-0f4b23f76c0d nodeName:}" failed. No retries permitted until 2026-07-24 19:11:22.984038892 +0000 UTC m=+1621.374990362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/59d7f887-7f7e-490a-8295-0f4b23f76c0d-maas-api-tls") pod "maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb" (UID: "59d7f887-7f7e-490a-8295-0f4b23f76c0d") : secret "maas-api-e2e-ait-ef3668b1-serving-cert" not found
Jul 24 19:11:22.484535 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.484517    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d7f887-7f7e-490a-8295-0f4b23f76c0d-openshift-service-ca\") pod \"maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb\" (UID: \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:22.494607 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.494584    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd587\" (UniqueName: \"kubernetes.io/projected/59d7f887-7f7e-490a-8295-0f4b23f76c0d-kube-api-access-kd587\") pod \"maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb\" (UID: \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:22.978970 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.978932    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"]
Jul 24 19:11:22.981563 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.981547    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"
Jul 24 19:11:22.984470 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.984452    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ef3668b1-dockercfg-k87lb\""
Jul 24 19:11:22.984578 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.984560    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ef3668b1\""
Jul 24 19:11:22.989383 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.989355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59d7f887-7f7e-490a-8295-0f4b23f76c0d-maas-api-tls\") pod \"maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb\" (UID: \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:22.992330 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.992300    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59d7f887-7f7e-490a-8295-0f4b23f76c0d-maas-api-tls\") pod \"maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb\" (UID: \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:22.992879 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:22.992864    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"]
Jul 24 19:11:23.090794 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.090750    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wmpd\" (UniqueName: \"kubernetes.io/projected/6315a5d9-82ed-4110-8b12-d3d926304066-kube-api-access-2wmpd\") pod \"payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh\" (UID: \"6315a5d9-82ed-4110-8b12-d3d926304066\") " pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"
Jul 24 19:11:23.090983 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.090884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6315a5d9-82ed-4110-8b12-d3d926304066-plugins-config-volume\") pod \"payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh\" (UID: \"6315a5d9-82ed-4110-8b12-d3d926304066\") " pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"
Jul 24 19:11:23.171028 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.170988    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:23.192251 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.192221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wmpd\" (UniqueName: \"kubernetes.io/projected/6315a5d9-82ed-4110-8b12-d3d926304066-kube-api-access-2wmpd\") pod \"payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh\" (UID: \"6315a5d9-82ed-4110-8b12-d3d926304066\") " pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"
Jul 24 19:11:23.192386 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.192301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6315a5d9-82ed-4110-8b12-d3d926304066-plugins-config-volume\") pod \"payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh\" (UID: \"6315a5d9-82ed-4110-8b12-d3d926304066\") " pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"
Jul 24 19:11:23.192989 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.192961    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6315a5d9-82ed-4110-8b12-d3d926304066-plugins-config-volume\") pod \"payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh\" (UID: \"6315a5d9-82ed-4110-8b12-d3d926304066\") " pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"
Jul 24 19:11:23.203694 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.203662    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wmpd\" (UniqueName: \"kubernetes.io/projected/6315a5d9-82ed-4110-8b12-d3d926304066-kube-api-access-2wmpd\") pod \"payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh\" (UID: \"6315a5d9-82ed-4110-8b12-d3d926304066\") " pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"
Jul 24 19:11:23.292779 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.292745    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"
Jul 24 19:11:23.317855 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.317830    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"]
Jul 24 19:11:23.318770 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:11:23.318731    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d7f887_7f7e_490a_8295_0f4b23f76c0d.slice/crio-5df04e75efeeea4da9207b339ad5ec9fa4609b9a434dfde2f81ab2a5694838fd WatchSource:0}: Error finding container 5df04e75efeeea4da9207b339ad5ec9fa4609b9a434dfde2f81ab2a5694838fd: Status 404 returned error can't find the container with id 5df04e75efeeea4da9207b339ad5ec9fa4609b9a434dfde2f81ab2a5694838fd
Jul 24 19:11:23.439608 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.439582    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"]
Jul 24 19:11:23.440628 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:11:23.440604    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6315a5d9_82ed_4110_8b12_d3d926304066.slice/crio-39ce5d60daf7e6222fba93e01314bfab721d98133e35b3733ac3e0e315d290c5 WatchSource:0}: Error finding container 39ce5d60daf7e6222fba93e01314bfab721d98133e35b3733ac3e0e315d290c5: Status 404 returned error can't find the container with id 39ce5d60daf7e6222fba93e01314bfab721d98133e35b3733ac3e0e315d290c5
Jul 24 19:11:23.810504 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.810457    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb" event={"ID":"59d7f887-7f7e-490a-8295-0f4b23f76c0d","Type":"ContainerStarted","Data":"780ae068f1b4a7aef6fcf98b0ee56aa2af924ba57f454fdcc23bc62d507b6dca"}
Jul 24 19:11:23.810690 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.810564    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:23.810690 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.810593    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb" event={"ID":"59d7f887-7f7e-490a-8295-0f4b23f76c0d","Type":"ContainerStarted","Data":"5df04e75efeeea4da9207b339ad5ec9fa4609b9a434dfde2f81ab2a5694838fd"}
Jul 24 19:11:23.812725 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.812690    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh" event={"ID":"6315a5d9-82ed-4110-8b12-d3d926304066","Type":"ContainerStarted","Data":"7022c7ea45178cd8883548f39c41d6d1d840f9c8fb7ce7f4149e1127d6986102"}
Jul 24 19:11:23.812886 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.812757    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"
Jul 24 19:11:23.812886 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.812782    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh" event={"ID":"6315a5d9-82ed-4110-8b12-d3d926304066","Type":"ContainerStarted","Data":"39ce5d60daf7e6222fba93e01314bfab721d98133e35b3733ac3e0e315d290c5"}
Jul 24 19:11:23.827097 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.827048    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb" podStartSLOduration=1.4891943890000001 podStartE2EDuration="1.827031391s" totalImagesPullingTime="337.837002ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:11:23.825846625 +0000 UTC m=+1622.216798111" watchObservedRunningTime="2026-07-24 19:11:23.827031391 +0000 UTC m=+1622.217982868"
Jul 24 19:11:23.841630 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:23.841550    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh" podStartSLOduration=1.841529177 podStartE2EDuration="1.841529177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:11:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:11:23.840874377 +0000 UTC m=+1622.231825855" watchObservedRunningTime="2026-07-24 19:11:23.841529177 +0000 UTC m=+1622.232480655"
Jul 24 19:11:24.502158 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.502135    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:11:24.604246 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.604208    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-data\") pod \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") "
Jul 24 19:11:24.604508 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.604268    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-workload-certs\") pod \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") "
Jul 24 19:11:24.604508 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.604288    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-envoy\") pod \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") "
Jul 24 19:11:24.604508 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.604312    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bf8mn\" (UniqueName: \"kubernetes.io/projected/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-kube-api-access-bf8mn\") pod \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") "
Jul 24 19:11:24.604508 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.604338    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-podinfo\") pod \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") "
Jul 24 19:11:24.604508 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.604381    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-credential-socket\") pod \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") "
Jul 24 19:11:24.604767 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.604553    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-workload-socket\") pod \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") "
Jul 24 19:11:24.604767 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.604603    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istiod-ca-cert\") pod \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") "
Jul 24 19:11:24.604767 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.604653    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "df14dbbe-f439-49f6-bcf1-5bd35f9cc385" (UID: "df14dbbe-f439-49f6-bcf1-5bd35f9cc385"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:11:24.604767 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.604666    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-token\") pod \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\" (UID: \"df14dbbe-f439-49f6-bcf1-5bd35f9cc385\") "
Jul 24 19:11:24.605009 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.604984    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-workload-certs\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:24.605247 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.605031    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "df14dbbe-f439-49f6-bcf1-5bd35f9cc385" (UID: "df14dbbe-f439-49f6-bcf1-5bd35f9cc385"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:11:24.605483 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.605459    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "df14dbbe-f439-49f6-bcf1-5bd35f9cc385" (UID: "df14dbbe-f439-49f6-bcf1-5bd35f9cc385"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:11:24.605483 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.605474    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-data" (OuterVolumeSpecName: "istio-data") pod "df14dbbe-f439-49f6-bcf1-5bd35f9cc385" (UID: "df14dbbe-f439-49f6-bcf1-5bd35f9cc385"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:11:24.605633 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.605554    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "df14dbbe-f439-49f6-bcf1-5bd35f9cc385" (UID: "df14dbbe-f439-49f6-bcf1-5bd35f9cc385"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:11:24.606841 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.606818    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-token" (OuterVolumeSpecName: "istio-token") pod "df14dbbe-f439-49f6-bcf1-5bd35f9cc385" (UID: "df14dbbe-f439-49f6-bcf1-5bd35f9cc385"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:11:24.606841 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.606824    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "df14dbbe-f439-49f6-bcf1-5bd35f9cc385" (UID: "df14dbbe-f439-49f6-bcf1-5bd35f9cc385"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:11:24.606949 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.606834    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "df14dbbe-f439-49f6-bcf1-5bd35f9cc385" (UID: "df14dbbe-f439-49f6-bcf1-5bd35f9cc385"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 19:11:24.606949 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.606859    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-kube-api-access-bf8mn" (OuterVolumeSpecName: "kube-api-access-bf8mn") pod "df14dbbe-f439-49f6-bcf1-5bd35f9cc385" (UID: "df14dbbe-f439-49f6-bcf1-5bd35f9cc385"). InnerVolumeSpecName "kube-api-access-bf8mn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:11:24.705897 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.705863    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-workload-socket\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:24.705897 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.705895    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istiod-ca-cert\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:24.705897 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.705906    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-token\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:24.706108 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.705926    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-data\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:24.706108 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.705934    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-envoy\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:24.706108 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.705942    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bf8mn\" (UniqueName: \"kubernetes.io/projected/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-kube-api-access-bf8mn\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:24.706108 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.705953    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-istio-podinfo\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:24.706108 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.705965    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df14dbbe-f439-49f6-bcf1-5bd35f9cc385-credential-socket\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:24.819696 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.819659    2563 generic.go:356] "Generic (PLEG): container finished" podID="df14dbbe-f439-49f6-bcf1-5bd35f9cc385" containerID="ed5acb59806ffda3c0160de1860492fbc1b695d31d22450bd8d43d2261281a95" exitCode=0
Jul 24 19:11:24.819867 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.819708    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"
Jul 24 19:11:24.819867 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.819737    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs" event={"ID":"df14dbbe-f439-49f6-bcf1-5bd35f9cc385","Type":"ContainerDied","Data":"ed5acb59806ffda3c0160de1860492fbc1b695d31d22450bd8d43d2261281a95"}
Jul 24 19:11:24.819867 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.819798    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs" event={"ID":"df14dbbe-f439-49f6-bcf1-5bd35f9cc385","Type":"ContainerDied","Data":"ee520cffb1eead78c3b2eab9c89a0d2c6758560c64f9c5e7d5f67898380cc41a"}
Jul 24 19:11:24.819867 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.819828    2563 scope.go:117] "RemoveContainer" containerID="ed5acb59806ffda3c0160de1860492fbc1b695d31d22450bd8d43d2261281a95"
Jul 24 19:11:24.830238 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.830210    2563 scope.go:117] "RemoveContainer" containerID="75c691731c274c8f68f57da2893fec2e23e1a4f1ae0f4ff3aa4a0a5dc94259db"
Jul 24 19:11:24.839038 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.839016    2563 scope.go:117] "RemoveContainer" containerID="ed5acb59806ffda3c0160de1860492fbc1b695d31d22450bd8d43d2261281a95"
Jul 24 19:11:24.839632 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:11:24.839497    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed5acb59806ffda3c0160de1860492fbc1b695d31d22450bd8d43d2261281a95\": container with ID starting with ed5acb59806ffda3c0160de1860492fbc1b695d31d22450bd8d43d2261281a95 not found: ID does not exist" containerID="ed5acb59806ffda3c0160de1860492fbc1b695d31d22450bd8d43d2261281a95"
Jul 24 19:11:24.839632 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.839538    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed5acb59806ffda3c0160de1860492fbc1b695d31d22450bd8d43d2261281a95"} err="failed to get container status \"ed5acb59806ffda3c0160de1860492fbc1b695d31d22450bd8d43d2261281a95\": rpc error: code = NotFound desc = could not find container \"ed5acb59806ffda3c0160de1860492fbc1b695d31d22450bd8d43d2261281a95\": container with ID starting with ed5acb59806ffda3c0160de1860492fbc1b695d31d22450bd8d43d2261281a95 not found: ID does not exist"
Jul 24 19:11:24.839632 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.839565    2563 scope.go:117] "RemoveContainer" containerID="75c691731c274c8f68f57da2893fec2e23e1a4f1ae0f4ff3aa4a0a5dc94259db"
Jul 24 19:11:24.839963 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:11:24.839942    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75c691731c274c8f68f57da2893fec2e23e1a4f1ae0f4ff3aa4a0a5dc94259db\": container with ID starting with 75c691731c274c8f68f57da2893fec2e23e1a4f1ae0f4ff3aa4a0a5dc94259db not found: ID does not exist" containerID="75c691731c274c8f68f57da2893fec2e23e1a4f1ae0f4ff3aa4a0a5dc94259db"
Jul 24 19:11:24.840008 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.839972    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c691731c274c8f68f57da2893fec2e23e1a4f1ae0f4ff3aa4a0a5dc94259db"} err="failed to get container status \"75c691731c274c8f68f57da2893fec2e23e1a4f1ae0f4ff3aa4a0a5dc94259db\": rpc error: code = NotFound desc = could not find container \"75c691731c274c8f68f57da2893fec2e23e1a4f1ae0f4ff3aa4a0a5dc94259db\": container with ID starting with 75c691731c274c8f68f57da2893fec2e23e1a4f1ae0f4ff3aa4a0a5dc94259db not found: ID does not exist"
Jul 24 19:11:24.842560 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.842535    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"]
Jul 24 19:11:24.845912 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:24.845891    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-e0dcb0ab-gw-openshift-default-5cc44cf9cb-rxzcs"]
Jul 24 19:11:26.204368 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:26.204337    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df14dbbe-f439-49f6-bcf1-5bd35f9cc385" path="/var/lib/kubelet/pods/df14dbbe-f439-49f6-bcf1-5bd35f9cc385/volumes"
Jul 24 19:11:29.457664 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:29.457631    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:11:29.825751 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:29.825721    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:34.822573 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:34.822453    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"
Jul 24 19:11:37.302468 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:37.302431    2563 patch_prober.go:28] interesting pod/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": read tcp 10.134.0.2:47222->10.134.0.35:15021: read: connection reset by peer" start-of-body=
Jul 24 19:11:37.302887 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:37.302514    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd" podUID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": read tcp 10.134.0.2:47222->10.134.0.35:15021: read: connection reset by peer"
Jul 24 19:11:37.886502 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:37.886467    2563 generic.go:356] "Generic (PLEG): container finished" podID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" containerID="2e24e6e65803940d00b9816ceb7d898b6a82b2e78c75701c1bb060ee909bff17" exitCode=0
Jul 24 19:11:37.886682 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:37.886541    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd" event={"ID":"05a69c7b-065e-4ce7-ac53-6e809d61a5af","Type":"ContainerDied","Data":"2e24e6e65803940d00b9816ceb7d898b6a82b2e78c75701c1bb060ee909bff17"}
Jul 24 19:11:37.886947 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:37.886932    2563 scope.go:117] "RemoveContainer" containerID="2e24e6e65803940d00b9816ceb7d898b6a82b2e78c75701c1bb060ee909bff17"
Jul 24 19:11:37.887933 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:37.887906    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:11:37.887983 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:37.887972    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:11:37.888046 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:37.888004    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:11:38.893737 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:38.893700    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd" event={"ID":"05a69c7b-065e-4ce7-ac53-6e809d61a5af","Type":"ContainerStarted","Data":"9d8349c8dddc02e556d5ed853f6bef44e8f5abd390dd3c391e6ee1d48478ba14"}
Jul 24 19:11:39.353738 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:39.353701    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:39.353930 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:39.353873    2563 patch_prober.go:28] interesting pod/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 24 19:11:39.353930 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:39.353917    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd" podUID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 24 19:11:40.353966 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:40.353918    2563 patch_prober.go:28] interesting pod/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 24 19:11:40.354497 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:40.353985    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd" podUID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 24 19:11:41.354605 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:41.354565    2563 patch_prober.go:28] interesting pod/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 24 19:11:41.355019 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:41.354625    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd" podUID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 24 19:11:42.354249 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:42.354205    2563 patch_prober.go:28] interesting pod/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 24 19:11:42.354511 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:42.354275    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd" podUID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 24 19:11:43.357714 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:43.357684    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:43.358118 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:43.358069    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:43.358768 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:43.358740    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:11:58.538814 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.538776    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"]
Jul 24 19:11:58.539300 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.539032    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb" podUID="59d7f887-7f7e-490a-8295-0f4b23f76c0d" containerName="maas-api" containerID="cri-o://780ae068f1b4a7aef6fcf98b0ee56aa2af924ba57f454fdcc23bc62d507b6dca" gracePeriod=30
Jul 24 19:11:58.598897 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.598862    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"]
Jul 24 19:11:58.599387 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.599334    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh" podUID="6315a5d9-82ed-4110-8b12-d3d926304066" containerName="payload-processing" containerID="cri-o://7022c7ea45178cd8883548f39c41d6d1d840f9c8fb7ce7f4149e1127d6986102" gracePeriod=30
Jul 24 19:11:58.846543 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.846518    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:58.880297 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.880267    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"
Jul 24 19:11:58.902981 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.902952    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d7f887-7f7e-490a-8295-0f4b23f76c0d-openshift-service-ca\") pod \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\" (UID: \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\") "
Jul 24 19:11:58.903151 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.903033    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kd587\" (UniqueName: \"kubernetes.io/projected/59d7f887-7f7e-490a-8295-0f4b23f76c0d-kube-api-access-kd587\") pod \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\" (UID: \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\") "
Jul 24 19:11:58.903151 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.903094    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59d7f887-7f7e-490a-8295-0f4b23f76c0d-maas-api-tls\") pod \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\" (UID: \"59d7f887-7f7e-490a-8295-0f4b23f76c0d\") "
Jul 24 19:11:58.903335 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.903311    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59d7f887-7f7e-490a-8295-0f4b23f76c0d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "59d7f887-7f7e-490a-8295-0f4b23f76c0d" (UID: "59d7f887-7f7e-490a-8295-0f4b23f76c0d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:11:58.905246 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.905220    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59d7f887-7f7e-490a-8295-0f4b23f76c0d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "59d7f887-7f7e-490a-8295-0f4b23f76c0d" (UID: "59d7f887-7f7e-490a-8295-0f4b23f76c0d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:11:58.905374 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.905307    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59d7f887-7f7e-490a-8295-0f4b23f76c0d-kube-api-access-kd587" (OuterVolumeSpecName: "kube-api-access-kd587") pod "59d7f887-7f7e-490a-8295-0f4b23f76c0d" (UID: "59d7f887-7f7e-490a-8295-0f4b23f76c0d"). InnerVolumeSpecName "kube-api-access-kd587". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:11:58.991968 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.991933    2563 generic.go:356] "Generic (PLEG): container finished" podID="59d7f887-7f7e-490a-8295-0f4b23f76c0d" containerID="780ae068f1b4a7aef6fcf98b0ee56aa2af924ba57f454fdcc23bc62d507b6dca" exitCode=0
Jul 24 19:11:58.992161 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.991981    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"
Jul 24 19:11:58.992161 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.992026    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb" event={"ID":"59d7f887-7f7e-490a-8295-0f4b23f76c0d","Type":"ContainerDied","Data":"780ae068f1b4a7aef6fcf98b0ee56aa2af924ba57f454fdcc23bc62d507b6dca"}
Jul 24 19:11:58.992161 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.992089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb" event={"ID":"59d7f887-7f7e-490a-8295-0f4b23f76c0d","Type":"ContainerDied","Data":"5df04e75efeeea4da9207b339ad5ec9fa4609b9a434dfde2f81ab2a5694838fd"}
Jul 24 19:11:58.992161 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.992123    2563 scope.go:117] "RemoveContainer" containerID="780ae068f1b4a7aef6fcf98b0ee56aa2af924ba57f454fdcc23bc62d507b6dca"
Jul 24 19:11:58.993425 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.993367    2563 generic.go:356] "Generic (PLEG): container finished" podID="6315a5d9-82ed-4110-8b12-d3d926304066" containerID="7022c7ea45178cd8883548f39c41d6d1d840f9c8fb7ce7f4149e1127d6986102" exitCode=0
Jul 24 19:11:58.993425 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.993416    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"
Jul 24 19:11:58.993671 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.993425    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh" event={"ID":"6315a5d9-82ed-4110-8b12-d3d926304066","Type":"ContainerDied","Data":"7022c7ea45178cd8883548f39c41d6d1d840f9c8fb7ce7f4149e1127d6986102"}
Jul 24 19:11:58.993671 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:58.993471    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh" event={"ID":"6315a5d9-82ed-4110-8b12-d3d926304066","Type":"ContainerDied","Data":"39ce5d60daf7e6222fba93e01314bfab721d98133e35b3733ac3e0e315d290c5"}
Jul 24 19:11:59.001137 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.001121    2563 scope.go:117] "RemoveContainer" containerID="780ae068f1b4a7aef6fcf98b0ee56aa2af924ba57f454fdcc23bc62d507b6dca"
Jul 24 19:11:59.001433 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:11:59.001413    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"780ae068f1b4a7aef6fcf98b0ee56aa2af924ba57f454fdcc23bc62d507b6dca\": container with ID starting with 780ae068f1b4a7aef6fcf98b0ee56aa2af924ba57f454fdcc23bc62d507b6dca not found: ID does not exist" containerID="780ae068f1b4a7aef6fcf98b0ee56aa2af924ba57f454fdcc23bc62d507b6dca"
Jul 24 19:11:59.001473 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.001442    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"780ae068f1b4a7aef6fcf98b0ee56aa2af924ba57f454fdcc23bc62d507b6dca"} err="failed to get container status \"780ae068f1b4a7aef6fcf98b0ee56aa2af924ba57f454fdcc23bc62d507b6dca\": rpc error: code = NotFound desc = could not find container \"780ae068f1b4a7aef6fcf98b0ee56aa2af924ba57f454fdcc23bc62d507b6dca\": container with ID starting with 780ae068f1b4a7aef6fcf98b0ee56aa2af924ba57f454fdcc23bc62d507b6dca not found: ID does not exist"
Jul 24 19:11:59.001473 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.001459    2563 scope.go:117] "RemoveContainer" containerID="7022c7ea45178cd8883548f39c41d6d1d840f9c8fb7ce7f4149e1127d6986102"
Jul 24 19:11:59.003604 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.003582    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wmpd\" (UniqueName: \"kubernetes.io/projected/6315a5d9-82ed-4110-8b12-d3d926304066-kube-api-access-2wmpd\") pod \"6315a5d9-82ed-4110-8b12-d3d926304066\" (UID: \"6315a5d9-82ed-4110-8b12-d3d926304066\") "
Jul 24 19:11:59.003671 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.003621    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6315a5d9-82ed-4110-8b12-d3d926304066-plugins-config-volume\") pod \"6315a5d9-82ed-4110-8b12-d3d926304066\" (UID: \"6315a5d9-82ed-4110-8b12-d3d926304066\") "
Jul 24 19:11:59.003828 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.003814    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kd587\" (UniqueName: \"kubernetes.io/projected/59d7f887-7f7e-490a-8295-0f4b23f76c0d-kube-api-access-kd587\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:59.003898 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.003830    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59d7f887-7f7e-490a-8295-0f4b23f76c0d-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:59.003898 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.003881    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d7f887-7f7e-490a-8295-0f4b23f76c0d-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:59.004026 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.004005    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6315a5d9-82ed-4110-8b12-d3d926304066-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6315a5d9-82ed-4110-8b12-d3d926304066" (UID: "6315a5d9-82ed-4110-8b12-d3d926304066"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:11:59.005691 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.005671    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6315a5d9-82ed-4110-8b12-d3d926304066-kube-api-access-2wmpd" (OuterVolumeSpecName: "kube-api-access-2wmpd") pod "6315a5d9-82ed-4110-8b12-d3d926304066" (UID: "6315a5d9-82ed-4110-8b12-d3d926304066"). InnerVolumeSpecName "kube-api-access-2wmpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:11:59.009590 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.009570    2563 scope.go:117] "RemoveContainer" containerID="7022c7ea45178cd8883548f39c41d6d1d840f9c8fb7ce7f4149e1127d6986102"
Jul 24 19:11:59.009910 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:11:59.009888    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7022c7ea45178cd8883548f39c41d6d1d840f9c8fb7ce7f4149e1127d6986102\": container with ID starting with 7022c7ea45178cd8883548f39c41d6d1d840f9c8fb7ce7f4149e1127d6986102 not found: ID does not exist" containerID="7022c7ea45178cd8883548f39c41d6d1d840f9c8fb7ce7f4149e1127d6986102"
Jul 24 19:11:59.009988 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.009917    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7022c7ea45178cd8883548f39c41d6d1d840f9c8fb7ce7f4149e1127d6986102"} err="failed to get container status \"7022c7ea45178cd8883548f39c41d6d1d840f9c8fb7ce7f4149e1127d6986102\": rpc error: code = NotFound desc = could not find container \"7022c7ea45178cd8883548f39c41d6d1d840f9c8fb7ce7f4149e1127d6986102\": container with ID starting with 7022c7ea45178cd8883548f39c41d6d1d840f9c8fb7ce7f4149e1127d6986102 not found: ID does not exist"
Jul 24 19:11:59.031241 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.031207    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"]
Jul 24 19:11:59.033094 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.033076    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef3668b1-5956c58c5d-b57qb"]
Jul 24 19:11:59.104788 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.104702    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wmpd\" (UniqueName: \"kubernetes.io/projected/6315a5d9-82ed-4110-8b12-d3d926304066-kube-api-access-2wmpd\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:59.104788 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.104745    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6315a5d9-82ed-4110-8b12-d3d926304066-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:11:59.316447 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.316412    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"]
Jul 24 19:11:59.322151 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:11:59.322122    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ef3668b1-7444646d78-p6swh"]
Jul 24 19:12:00.204309 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:00.204273    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59d7f887-7f7e-490a-8295-0f4b23f76c0d" path="/var/lib/kubelet/pods/59d7f887-7f7e-490a-8295-0f4b23f76c0d/volumes"
Jul 24 19:12:00.204692 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:00.204679    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6315a5d9-82ed-4110-8b12-d3d926304066" path="/var/lib/kubelet/pods/6315a5d9-82ed-4110-8b12-d3d926304066/volumes"
Jul 24 19:12:01.150408 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:01.150354    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:12:04.217819 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:04.217789    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"]
Jul 24 19:12:04.218241 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:04.217990    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd" podUID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" containerName="istio-proxy" containerID="cri-o://9d8349c8dddc02e556d5ed853f6bef44e8f5abd390dd3c391e6ee1d48478ba14" gracePeriod=30
Jul 24 19:12:07.813054 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.813021    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"]
Jul 24 19:12:07.813627 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.813468    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59d7f887-7f7e-490a-8295-0f4b23f76c0d" containerName="maas-api"
Jul 24 19:12:07.813627 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.813488    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="59d7f887-7f7e-490a-8295-0f4b23f76c0d" containerName="maas-api"
Jul 24 19:12:07.813627 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.813504    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df14dbbe-f439-49f6-bcf1-5bd35f9cc385" containerName="istio-proxy"
Jul 24 19:12:07.813627 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.813512    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="df14dbbe-f439-49f6-bcf1-5bd35f9cc385" containerName="istio-proxy"
Jul 24 19:12:07.813627 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.813521    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6315a5d9-82ed-4110-8b12-d3d926304066" containerName="payload-processing"
Jul 24 19:12:07.813627 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.813530    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6315a5d9-82ed-4110-8b12-d3d926304066" containerName="payload-processing"
Jul 24 19:12:07.813627 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.813540    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df14dbbe-f439-49f6-bcf1-5bd35f9cc385" containerName="istio-proxy"
Jul 24 19:12:07.813627 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.813550    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="df14dbbe-f439-49f6-bcf1-5bd35f9cc385" containerName="istio-proxy"
Jul 24 19:12:07.814034 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.813633    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="df14dbbe-f439-49f6-bcf1-5bd35f9cc385" containerName="istio-proxy"
Jul 24 19:12:07.814034 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.813648    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="59d7f887-7f7e-490a-8295-0f4b23f76c0d" containerName="maas-api"
Jul 24 19:12:07.814034 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.813656    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="df14dbbe-f439-49f6-bcf1-5bd35f9cc385" containerName="istio-proxy"
Jul 24 19:12:07.814034 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.813665    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6315a5d9-82ed-4110-8b12-d3d926304066" containerName="payload-processing"
Jul 24 19:12:07.816485 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.816465    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:07.818986 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.818966    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cvjqn\""
Jul 24 19:12:07.819092 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.818965    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-73fbe69e-serving-cert\""
Jul 24 19:12:07.819092 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.819024    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 19:12:07.819837 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.819821    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 19:12:07.833683 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.833655    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"]
Jul 24 19:12:07.985599 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.985562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89aea030-24fb-4816-8da9-dacbea55ec34-maas-api-tls\") pod \"maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm\" (UID: \"89aea030-24fb-4816-8da9-dacbea55ec34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:07.985599 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.985603    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89aea030-24fb-4816-8da9-dacbea55ec34-openshift-service-ca\") pod \"maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm\" (UID: \"89aea030-24fb-4816-8da9-dacbea55ec34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:07.985854 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:07.985664    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjz78\" (UniqueName: \"kubernetes.io/projected/89aea030-24fb-4816-8da9-dacbea55ec34-kube-api-access-fjz78\") pod \"maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm\" (UID: \"89aea030-24fb-4816-8da9-dacbea55ec34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:08.086366 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.086268    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89aea030-24fb-4816-8da9-dacbea55ec34-maas-api-tls\") pod \"maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm\" (UID: \"89aea030-24fb-4816-8da9-dacbea55ec34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:08.086366 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.086307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89aea030-24fb-4816-8da9-dacbea55ec34-openshift-service-ca\") pod \"maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm\" (UID: \"89aea030-24fb-4816-8da9-dacbea55ec34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:08.086366 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.086337    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjz78\" (UniqueName: \"kubernetes.io/projected/89aea030-24fb-4816-8da9-dacbea55ec34-kube-api-access-fjz78\") pod \"maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm\" (UID: \"89aea030-24fb-4816-8da9-dacbea55ec34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:08.087541 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.087501    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89aea030-24fb-4816-8da9-dacbea55ec34-openshift-service-ca\") pod \"maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm\" (UID: \"89aea030-24fb-4816-8da9-dacbea55ec34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:08.093036 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.093010    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89aea030-24fb-4816-8da9-dacbea55ec34-maas-api-tls\") pod \"maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm\" (UID: \"89aea030-24fb-4816-8da9-dacbea55ec34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:08.095742 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.095721    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjz78\" (UniqueName: \"kubernetes.io/projected/89aea030-24fb-4816-8da9-dacbea55ec34-kube-api-access-fjz78\") pod \"maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm\" (UID: \"89aea030-24fb-4816-8da9-dacbea55ec34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:08.126761 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.126732    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:08.263653 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.263576    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"]
Jul 24 19:12:08.265884 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:12:08.265849    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89aea030_24fb_4816_8da9_dacbea55ec34.slice/crio-692b3ec4c64a933a86cb15411146b701526b0ed94283578dd3fdda8250443636 WatchSource:0}: Error finding container 692b3ec4c64a933a86cb15411146b701526b0ed94283578dd3fdda8250443636: Status 404 returned error can't find the container with id 692b3ec4c64a933a86cb15411146b701526b0ed94283578dd3fdda8250443636
Jul 24 19:12:08.559111 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.559080    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"]
Jul 24 19:12:08.561889 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.561873    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"
Jul 24 19:12:08.564689 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.564671    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-73fbe69e-dockercfg-9bf2b\""
Jul 24 19:12:08.564860 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.564841    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-73fbe69e\""
Jul 24 19:12:08.575889 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.575865    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"]
Jul 24 19:12:08.691749 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.691714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hqlh\" (UniqueName: \"kubernetes.io/projected/7d94ff43-4909-4a49-a016-89b3f2cbbed6-kube-api-access-8hqlh\") pod \"payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl\" (UID: \"7d94ff43-4909-4a49-a016-89b3f2cbbed6\") " pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"
Jul 24 19:12:08.691919 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.691799    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d94ff43-4909-4a49-a016-89b3f2cbbed6-plugins-config-volume\") pod \"payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl\" (UID: \"7d94ff43-4909-4a49-a016-89b3f2cbbed6\") " pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"
Jul 24 19:12:08.792274 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.792242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hqlh\" (UniqueName: \"kubernetes.io/projected/7d94ff43-4909-4a49-a016-89b3f2cbbed6-kube-api-access-8hqlh\") pod \"payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl\" (UID: \"7d94ff43-4909-4a49-a016-89b3f2cbbed6\") " pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"
Jul 24 19:12:08.792453 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.792286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d94ff43-4909-4a49-a016-89b3f2cbbed6-plugins-config-volume\") pod \"payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl\" (UID: \"7d94ff43-4909-4a49-a016-89b3f2cbbed6\") " pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"
Jul 24 19:12:08.792945 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.792923    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d94ff43-4909-4a49-a016-89b3f2cbbed6-plugins-config-volume\") pod \"payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl\" (UID: \"7d94ff43-4909-4a49-a016-89b3f2cbbed6\") " pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"
Jul 24 19:12:08.808717 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.808692    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hqlh\" (UniqueName: \"kubernetes.io/projected/7d94ff43-4909-4a49-a016-89b3f2cbbed6-kube-api-access-8hqlh\") pod \"payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl\" (UID: \"7d94ff43-4909-4a49-a016-89b3f2cbbed6\") " pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"
Jul 24 19:12:08.871887 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:08.871860    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"
Jul 24 19:12:09.003930 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.003907    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"]
Jul 24 19:12:09.005891 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:12:09.005865    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d94ff43_4909_4a49_a016_89b3f2cbbed6.slice/crio-744b7729d56f76d413a43779032851f48c1bbf136de120b1248b400dcd4ea46a WatchSource:0}: Error finding container 744b7729d56f76d413a43779032851f48c1bbf136de120b1248b400dcd4ea46a: Status 404 returned error can't find the container with id 744b7729d56f76d413a43779032851f48c1bbf136de120b1248b400dcd4ea46a
Jul 24 19:12:09.048261 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.048234    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl" event={"ID":"7d94ff43-4909-4a49-a016-89b3f2cbbed6","Type":"ContainerStarted","Data":"744b7729d56f76d413a43779032851f48c1bbf136de120b1248b400dcd4ea46a"}
Jul 24 19:12:09.049702 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.049677    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm" event={"ID":"89aea030-24fb-4816-8da9-dacbea55ec34","Type":"ContainerStarted","Data":"6ed73b77f9dbb7459e3baa24cb7090940b81b041b6a1e76f0537fd710ee9843e"}
Jul 24 19:12:09.049818 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.049727    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm" event={"ID":"89aea030-24fb-4816-8da9-dacbea55ec34","Type":"ContainerStarted","Data":"692b3ec4c64a933a86cb15411146b701526b0ed94283578dd3fdda8250443636"}
Jul 24 19:12:09.049818 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.049768    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:09.071776 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.071734    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm" podStartSLOduration=1.537087313 podStartE2EDuration="2.071718377s" totalImagesPullingTime="534.631064ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:12:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:12:09.069974671 +0000 UTC m=+1667.460926160" watchObservedRunningTime="2026-07-24 19:12:09.071718377 +0000 UTC m=+1667.462669854"
Jul 24 19:12:09.467495 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.467472    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:12:09.598464 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.598360    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bc6h\" (UniqueName: \"kubernetes.io/projected/05a69c7b-065e-4ce7-ac53-6e809d61a5af-kube-api-access-4bc6h\") pod \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") "
Jul 24 19:12:09.598464 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.598422    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istiod-ca-cert\") pod \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") "
Jul 24 19:12:09.598464 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.598441    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-token\") pod \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") "
Jul 24 19:12:09.598464 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.598458    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-data\") pod \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") "
Jul 24 19:12:09.598805 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.598483    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-workload-socket\") pod \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") "
Jul 24 19:12:09.598805 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.598506    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-podinfo\") pod \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") "
Jul 24 19:12:09.598805 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.598558    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-envoy\") pod \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") "
Jul 24 19:12:09.598805 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.598593    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-credential-socket\") pod \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") "
Jul 24 19:12:09.598805 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.598623    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-workload-certs\") pod \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\" (UID: \"05a69c7b-065e-4ce7-ac53-6e809d61a5af\") "
Jul 24 19:12:09.599270 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.598946    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "05a69c7b-065e-4ce7-ac53-6e809d61a5af" (UID: "05a69c7b-065e-4ce7-ac53-6e809d61a5af"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:12:09.599270 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.599013    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "05a69c7b-065e-4ce7-ac53-6e809d61a5af" (UID: "05a69c7b-065e-4ce7-ac53-6e809d61a5af"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:12:09.599528 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.599450    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "05a69c7b-065e-4ce7-ac53-6e809d61a5af" (UID: "05a69c7b-065e-4ce7-ac53-6e809d61a5af"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:12:09.599528 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.599474    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "05a69c7b-065e-4ce7-ac53-6e809d61a5af" (UID: "05a69c7b-065e-4ce7-ac53-6e809d61a5af"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:12:09.599957 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.599929    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-data" (OuterVolumeSpecName: "istio-data") pod "05a69c7b-065e-4ce7-ac53-6e809d61a5af" (UID: "05a69c7b-065e-4ce7-ac53-6e809d61a5af"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:12:09.600906 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.600883    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05a69c7b-065e-4ce7-ac53-6e809d61a5af-kube-api-access-4bc6h" (OuterVolumeSpecName: "kube-api-access-4bc6h") pod "05a69c7b-065e-4ce7-ac53-6e809d61a5af" (UID: "05a69c7b-065e-4ce7-ac53-6e809d61a5af"). InnerVolumeSpecName "kube-api-access-4bc6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:12:09.600906 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.600889    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "05a69c7b-065e-4ce7-ac53-6e809d61a5af" (UID: "05a69c7b-065e-4ce7-ac53-6e809d61a5af"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 19:12:09.601050 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.600966    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-token" (OuterVolumeSpecName: "istio-token") pod "05a69c7b-065e-4ce7-ac53-6e809d61a5af" (UID: "05a69c7b-065e-4ce7-ac53-6e809d61a5af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:12:09.601050 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.601020    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "05a69c7b-065e-4ce7-ac53-6e809d61a5af" (UID: "05a69c7b-065e-4ce7-ac53-6e809d61a5af"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:12:09.699650 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.699618    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-token\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:12:09.699650 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.699644    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-data\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:12:09.699650 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.699653    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-workload-socket\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:12:09.699868 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.699662    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-podinfo\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:12:09.699868 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.699670    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istio-envoy\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:12:09.699868 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.699679    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-credential-socket\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:12:09.699868 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.699687    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05a69c7b-065e-4ce7-ac53-6e809d61a5af-workload-certs\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:12:09.699868 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.699696    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bc6h\" (UniqueName: \"kubernetes.io/projected/05a69c7b-065e-4ce7-ac53-6e809d61a5af-kube-api-access-4bc6h\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:12:09.699868 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:09.699704    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05a69c7b-065e-4ce7-ac53-6e809d61a5af-istiod-ca-cert\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:12:10.057180 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.057139    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl" event={"ID":"7d94ff43-4909-4a49-a016-89b3f2cbbed6","Type":"ContainerStarted","Data":"e21fbf6d2e778f1686d177d3fc7a807c39b371572a380df24b118738637bf9ab"}
Jul 24 19:12:10.057669 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.057220    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"
Jul 24 19:12:10.059121 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.059098    2563 generic.go:356] "Generic (PLEG): container finished" podID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" containerID="9d8349c8dddc02e556d5ed853f6bef44e8f5abd390dd3c391e6ee1d48478ba14" exitCode=0
Jul 24 19:12:10.059269 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.059136    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd" event={"ID":"05a69c7b-065e-4ce7-ac53-6e809d61a5af","Type":"ContainerDied","Data":"9d8349c8dddc02e556d5ed853f6bef44e8f5abd390dd3c391e6ee1d48478ba14"}
Jul 24 19:12:10.059269 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.059147    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"
Jul 24 19:12:10.059269 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.059190    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd" event={"ID":"05a69c7b-065e-4ce7-ac53-6e809d61a5af","Type":"ContainerDied","Data":"c01d227878c69689887a847a4e40577ff4a48db583a332e6a57a0fb90a91ae7c"}
Jul 24 19:12:10.059269 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.059214    2563 scope.go:117] "RemoveContainer" containerID="9d8349c8dddc02e556d5ed853f6bef44e8f5abd390dd3c391e6ee1d48478ba14"
Jul 24 19:12:10.068007 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.067990    2563 scope.go:117] "RemoveContainer" containerID="2e24e6e65803940d00b9816ceb7d898b6a82b2e78c75701c1bb060ee909bff17"
Jul 24 19:12:10.073242 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.073196    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl" podStartSLOduration=2.073180899 podStartE2EDuration="2.073180899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:12:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:12:10.072486887 +0000 UTC m=+1668.463438365" watchObservedRunningTime="2026-07-24 19:12:10.073180899 +0000 UTC m=+1668.464132422"
Jul 24 19:12:10.076869 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.076853    2563 scope.go:117] "RemoveContainer" containerID="9d8349c8dddc02e556d5ed853f6bef44e8f5abd390dd3c391e6ee1d48478ba14"
Jul 24 19:12:10.077218 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:12:10.077199    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d8349c8dddc02e556d5ed853f6bef44e8f5abd390dd3c391e6ee1d48478ba14\": container with ID starting with 9d8349c8dddc02e556d5ed853f6bef44e8f5abd390dd3c391e6ee1d48478ba14 not found: ID does not exist" containerID="9d8349c8dddc02e556d5ed853f6bef44e8f5abd390dd3c391e6ee1d48478ba14"
Jul 24 19:12:10.077257 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.077227    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d8349c8dddc02e556d5ed853f6bef44e8f5abd390dd3c391e6ee1d48478ba14"} err="failed to get container status \"9d8349c8dddc02e556d5ed853f6bef44e8f5abd390dd3c391e6ee1d48478ba14\": rpc error: code = NotFound desc = could not find container \"9d8349c8dddc02e556d5ed853f6bef44e8f5abd390dd3c391e6ee1d48478ba14\": container with ID starting with 9d8349c8dddc02e556d5ed853f6bef44e8f5abd390dd3c391e6ee1d48478ba14 not found: ID does not exist"
Jul 24 19:12:10.077257 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.077243    2563 scope.go:117] "RemoveContainer" containerID="2e24e6e65803940d00b9816ceb7d898b6a82b2e78c75701c1bb060ee909bff17"
Jul 24 19:12:10.077534 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:12:10.077507    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e24e6e65803940d00b9816ceb7d898b6a82b2e78c75701c1bb060ee909bff17\": container with ID starting with 2e24e6e65803940d00b9816ceb7d898b6a82b2e78c75701c1bb060ee909bff17 not found: ID does not exist" containerID="2e24e6e65803940d00b9816ceb7d898b6a82b2e78c75701c1bb060ee909bff17"
Jul 24 19:12:10.077582 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.077539    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e24e6e65803940d00b9816ceb7d898b6a82b2e78c75701c1bb060ee909bff17"} err="failed to get container status \"2e24e6e65803940d00b9816ceb7d898b6a82b2e78c75701c1bb060ee909bff17\": rpc error: code = NotFound desc = could not find container \"2e24e6e65803940d00b9816ceb7d898b6a82b2e78c75701c1bb060ee909bff17\": container with ID starting with 2e24e6e65803940d00b9816ceb7d898b6a82b2e78c75701c1bb060ee909bff17 not found: ID does not exist"
Jul 24 19:12:10.086128 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.086106    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"]
Jul 24 19:12:10.094674 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.094648    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ef3668b1-openshift-default-67cc57f758-rw6gd"]
Jul 24 19:12:10.204437 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:10.204382    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" path="/var/lib/kubelet/pods/05a69c7b-065e-4ce7-ac53-6e809d61a5af/volumes"
Jul 24 19:12:12.655100 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:12.655063    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:12:15.064294 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:15.064266    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:21.068523 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:21.068490    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"
Jul 24 19:12:43.108887 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.108633    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"]
Jul 24 19:12:43.109520 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.108906    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm" podUID="89aea030-24fb-4816-8da9-dacbea55ec34" containerName="maas-api" containerID="cri-o://6ed73b77f9dbb7459e3baa24cb7090940b81b041b6a1e76f0537fd710ee9843e" gracePeriod=30
Jul 24 19:12:43.168042 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.168009    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"]
Jul 24 19:12:43.168276 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.168229    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl" podUID="7d94ff43-4909-4a49-a016-89b3f2cbbed6" containerName="payload-processing" containerID="cri-o://e21fbf6d2e778f1686d177d3fc7a807c39b371572a380df24b118738637bf9ab" gracePeriod=30
Jul 24 19:12:43.230039 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.230005    2563 generic.go:356] "Generic (PLEG): container finished" podID="89aea030-24fb-4816-8da9-dacbea55ec34" containerID="6ed73b77f9dbb7459e3baa24cb7090940b81b041b6a1e76f0537fd710ee9843e" exitCode=0
Jul 24 19:12:43.230196 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.230073    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm" event={"ID":"89aea030-24fb-4816-8da9-dacbea55ec34","Type":"ContainerDied","Data":"6ed73b77f9dbb7459e3baa24cb7090940b81b041b6a1e76f0537fd710ee9843e"}
Jul 24 19:12:43.407953 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.407931    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:43.438538 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.438504    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"
Jul 24 19:12:43.488906 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.488876    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjz78\" (UniqueName: \"kubernetes.io/projected/89aea030-24fb-4816-8da9-dacbea55ec34-kube-api-access-fjz78\") pod \"89aea030-24fb-4816-8da9-dacbea55ec34\" (UID: \"89aea030-24fb-4816-8da9-dacbea55ec34\") "
Jul 24 19:12:43.488906 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.488911    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hqlh\" (UniqueName: \"kubernetes.io/projected/7d94ff43-4909-4a49-a016-89b3f2cbbed6-kube-api-access-8hqlh\") pod \"7d94ff43-4909-4a49-a016-89b3f2cbbed6\" (UID: \"7d94ff43-4909-4a49-a016-89b3f2cbbed6\") "
Jul 24 19:12:43.489149 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.488937    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d94ff43-4909-4a49-a016-89b3f2cbbed6-plugins-config-volume\") pod \"7d94ff43-4909-4a49-a016-89b3f2cbbed6\" (UID: \"7d94ff43-4909-4a49-a016-89b3f2cbbed6\") "
Jul 24 19:12:43.489149 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.488983    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89aea030-24fb-4816-8da9-dacbea55ec34-maas-api-tls\") pod \"89aea030-24fb-4816-8da9-dacbea55ec34\" (UID: \"89aea030-24fb-4816-8da9-dacbea55ec34\") "
Jul 24 19:12:43.489149 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.489012    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89aea030-24fb-4816-8da9-dacbea55ec34-openshift-service-ca\") pod \"89aea030-24fb-4816-8da9-dacbea55ec34\" (UID: \"89aea030-24fb-4816-8da9-dacbea55ec34\") "
Jul 24 19:12:43.489381 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.489358    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d94ff43-4909-4a49-a016-89b3f2cbbed6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7d94ff43-4909-4a49-a016-89b3f2cbbed6" (UID: "7d94ff43-4909-4a49-a016-89b3f2cbbed6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:12:43.489485 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.489464    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89aea030-24fb-4816-8da9-dacbea55ec34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "89aea030-24fb-4816-8da9-dacbea55ec34" (UID: "89aea030-24fb-4816-8da9-dacbea55ec34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:12:43.491195 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.491175    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d94ff43-4909-4a49-a016-89b3f2cbbed6-kube-api-access-8hqlh" (OuterVolumeSpecName: "kube-api-access-8hqlh") pod "7d94ff43-4909-4a49-a016-89b3f2cbbed6" (UID: "7d94ff43-4909-4a49-a016-89b3f2cbbed6"). InnerVolumeSpecName "kube-api-access-8hqlh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:12:43.491613 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.491592    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89aea030-24fb-4816-8da9-dacbea55ec34-kube-api-access-fjz78" (OuterVolumeSpecName: "kube-api-access-fjz78") pod "89aea030-24fb-4816-8da9-dacbea55ec34" (UID: "89aea030-24fb-4816-8da9-dacbea55ec34"). InnerVolumeSpecName "kube-api-access-fjz78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:12:43.491613 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.491601    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89aea030-24fb-4816-8da9-dacbea55ec34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "89aea030-24fb-4816-8da9-dacbea55ec34" (UID: "89aea030-24fb-4816-8da9-dacbea55ec34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:12:43.589638 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.589602    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjz78\" (UniqueName: \"kubernetes.io/projected/89aea030-24fb-4816-8da9-dacbea55ec34-kube-api-access-fjz78\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:12:43.589638 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.589632    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hqlh\" (UniqueName: \"kubernetes.io/projected/7d94ff43-4909-4a49-a016-89b3f2cbbed6-kube-api-access-8hqlh\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:12:43.589638 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.589643    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d94ff43-4909-4a49-a016-89b3f2cbbed6-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:12:43.589869 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.589654    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89aea030-24fb-4816-8da9-dacbea55ec34-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:12:43.589869 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:43.589663    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89aea030-24fb-4816-8da9-dacbea55ec34-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:12:44.236059 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:44.236025    2563 generic.go:356] "Generic (PLEG): container finished" podID="7d94ff43-4909-4a49-a016-89b3f2cbbed6" containerID="e21fbf6d2e778f1686d177d3fc7a807c39b371572a380df24b118738637bf9ab" exitCode=0
Jul 24 19:12:44.236494 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:44.236068    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"
Jul 24 19:12:44.236494 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:44.236118    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl" event={"ID":"7d94ff43-4909-4a49-a016-89b3f2cbbed6","Type":"ContainerDied","Data":"e21fbf6d2e778f1686d177d3fc7a807c39b371572a380df24b118738637bf9ab"}
Jul 24 19:12:44.236494 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:44.236176    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl" event={"ID":"7d94ff43-4909-4a49-a016-89b3f2cbbed6","Type":"ContainerDied","Data":"744b7729d56f76d413a43779032851f48c1bbf136de120b1248b400dcd4ea46a"}
Jul 24 19:12:44.236494 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:44.236204    2563 scope.go:117] "RemoveContainer" containerID="e21fbf6d2e778f1686d177d3fc7a807c39b371572a380df24b118738637bf9ab"
Jul 24 19:12:44.237525 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:44.237492    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm" event={"ID":"89aea030-24fb-4816-8da9-dacbea55ec34","Type":"ContainerDied","Data":"692b3ec4c64a933a86cb15411146b701526b0ed94283578dd3fdda8250443636"}
Jul 24 19:12:44.237525 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:44.237514    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"
Jul 24 19:12:44.245055 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:44.245039    2563 scope.go:117] "RemoveContainer" containerID="e21fbf6d2e778f1686d177d3fc7a807c39b371572a380df24b118738637bf9ab"
Jul 24 19:12:44.245300 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:12:44.245279    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e21fbf6d2e778f1686d177d3fc7a807c39b371572a380df24b118738637bf9ab\": container with ID starting with e21fbf6d2e778f1686d177d3fc7a807c39b371572a380df24b118738637bf9ab not found: ID does not exist" containerID="e21fbf6d2e778f1686d177d3fc7a807c39b371572a380df24b118738637bf9ab"
Jul 24 19:12:44.245366 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:44.245313    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e21fbf6d2e778f1686d177d3fc7a807c39b371572a380df24b118738637bf9ab"} err="failed to get container status \"e21fbf6d2e778f1686d177d3fc7a807c39b371572a380df24b118738637bf9ab\": rpc error: code = NotFound desc = could not find container \"e21fbf6d2e778f1686d177d3fc7a807c39b371572a380df24b118738637bf9ab\": container with ID starting with e21fbf6d2e778f1686d177d3fc7a807c39b371572a380df24b118738637bf9ab not found: ID does not exist"
Jul 24 19:12:44.245366 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:44.245336    2563 scope.go:117] "RemoveContainer" containerID="6ed73b77f9dbb7459e3baa24cb7090940b81b041b6a1e76f0537fd710ee9843e"
Jul 24 19:12:44.259593 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:44.259571    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"]
Jul 24 19:12:44.262728 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:44.262706    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-73fbe69e-64459b777f-mxkvm"]
Jul 24 19:12:44.272812 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:44.272793    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"]
Jul 24 19:12:44.274977 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:44.274958    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-73fbe69e-b657f79f8-fggbl"]
Jul 24 19:12:46.147220 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:46.147172    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:12:46.204255 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:46.204212    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d94ff43-4909-4a49-a016-89b3f2cbbed6" path="/var/lib/kubelet/pods/7d94ff43-4909-4a49-a016-89b3f2cbbed6/volumes"
Jul 24 19:12:46.204694 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:12:46.204674    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89aea030-24fb-4816-8da9-dacbea55ec34" path="/var/lib/kubelet/pods/89aea030-24fb-4816-8da9-dacbea55ec34/volumes"
Jul 24 19:14:12.754028 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:12.753934    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:14:22.195693 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:22.195665    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 19:14:22.201071 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:22.201044    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 19:14:25.556353 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:25.556323    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:14:35.759130 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:35.759093    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:14:36.414439 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.414383    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"]
Jul 24 19:14:36.414709 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.414697    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" containerName="istio-proxy"
Jul 24 19:14:36.414765 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.414711    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" containerName="istio-proxy"
Jul 24 19:14:36.414765 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.414722    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" containerName="istio-proxy"
Jul 24 19:14:36.414765 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.414729    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" containerName="istio-proxy"
Jul 24 19:14:36.414765 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.414745    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d94ff43-4909-4a49-a016-89b3f2cbbed6" containerName="payload-processing"
Jul 24 19:14:36.414765 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.414759    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d94ff43-4909-4a49-a016-89b3f2cbbed6" containerName="payload-processing"
Jul 24 19:14:36.414916 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.414768    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89aea030-24fb-4816-8da9-dacbea55ec34" containerName="maas-api"
Jul 24 19:14:36.414916 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.414773    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="89aea030-24fb-4816-8da9-dacbea55ec34" containerName="maas-api"
Jul 24 19:14:36.414916 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.414820    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" containerName="istio-proxy"
Jul 24 19:14:36.414916 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.414827    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d94ff43-4909-4a49-a016-89b3f2cbbed6" containerName="payload-processing"
Jul 24 19:14:36.414916 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.414835    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="89aea030-24fb-4816-8da9-dacbea55ec34" containerName="maas-api"
Jul 24 19:14:36.414916 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.414840    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="05a69c7b-065e-4ce7-ac53-6e809d61a5af" containerName="istio-proxy"
Jul 24 19:14:36.417727 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.417711    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.420015 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.419995    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-16b0ae71-openshift-default-dockercfg-jwrpc\""
Jul 24 19:14:36.439187 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.439161    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"]
Jul 24 19:14:36.507324 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.507283    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-workload-socket\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.507545 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.507364    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-podinfo\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.507545 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.507415    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-workload-certs\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.507545 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.507439    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istiod-ca-cert\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.507545 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.507463    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-data\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.507545 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.507509    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-token\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.507768 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.507548    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-credential-socket\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.507768 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.507570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t5sd\" (UniqueName: \"kubernetes.io/projected/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-kube-api-access-9t5sd\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.507768 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.507600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-envoy\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.608812 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.608776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-podinfo\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.609010 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.608823    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-workload-certs\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.609010 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.608846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istiod-ca-cert\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.609010 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.608874    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-data\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.609010 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.608908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-token\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.609010 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.608959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-credential-socket\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.609010 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.608982    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t5sd\" (UniqueName: \"kubernetes.io/projected/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-kube-api-access-9t5sd\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.609010 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.609008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-envoy\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.609375 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.609056    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-workload-socket\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.609375 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.609254    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-workload-certs\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.609375 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.609308    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-data\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.609775 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.609750    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-workload-socket\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.609908 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.609780    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istiod-ca-cert\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.609908 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.609807    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-credential-socket\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.611819 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.611800    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-envoy\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.611920 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.611823    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-podinfo\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.617059 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.617032    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-token\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.617147 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.617035    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t5sd\" (UniqueName: \"kubernetes.io/projected/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-kube-api-access-9t5sd\") pod \"e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") " pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.728336 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.728241    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:36.864868 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.864842    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"]
Jul 24 19:14:36.866787 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:14:36.866757    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba66457d_d0d8_4c2c_9cb4_d98bc15308f3.slice/crio-9ba22db73b0cadbb79ff09de8a33a672267d2f0bd7704f8d2863766569298517 WatchSource:0}: Error finding container 9ba22db73b0cadbb79ff09de8a33a672267d2f0bd7704f8d2863766569298517: Status 404 returned error can't find the container with id 9ba22db73b0cadbb79ff09de8a33a672267d2f0bd7704f8d2863766569298517
Jul 24 19:14:36.869119 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.869085    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:14:36.869193 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.869151    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:14:36.869193 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:36.869179    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:14:37.778083 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:37.778050    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n" event={"ID":"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3","Type":"ContainerStarted","Data":"c89fd0d6543621aae642d31c91ec43b01e1b07acfa8df25306de40ac692e39d7"}
Jul 24 19:14:37.778083 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:37.778102    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n" event={"ID":"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3","Type":"ContainerStarted","Data":"9ba22db73b0cadbb79ff09de8a33a672267d2f0bd7704f8d2863766569298517"}
Jul 24 19:14:37.799159 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:37.799111    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n" podStartSLOduration=1.799095458 podStartE2EDuration="1.799095458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:14:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:14:37.796824172 +0000 UTC m=+1816.187775669" watchObservedRunningTime="2026-07-24 19:14:37.799095458 +0000 UTC m=+1816.190046935"
Jul 24 19:14:38.728962 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:38.728926    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:38.733645 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:38.733618    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:38.782795 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:38.782766    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:38.783644 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:38.783626    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:39.886429 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:39.886383    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"]
Jul 24 19:14:39.889709 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:39.889687    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:14:39.893234 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:39.893213    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 19:14:39.894006 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:39.893989    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 19:14:39.894110 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:39.894012    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-16b0ae71-serving-cert\""
Jul 24 19:14:39.894110 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:39.894014    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cvjqn\""
Jul 24 19:14:39.904209 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:39.904187    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"]
Jul 24 19:14:39.940045 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:39.940011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94d461e4-7e7b-4add-94c2-aa6bacf83129-maas-api-tls\") pod \"maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8\" (UID: \"94d461e4-7e7b-4add-94c2-aa6bacf83129\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:14:39.940200 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:39.940064    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d461e4-7e7b-4add-94c2-aa6bacf83129-openshift-service-ca\") pod \"maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8\" (UID: \"94d461e4-7e7b-4add-94c2-aa6bacf83129\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:14:39.940200 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:39.940166    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwcfv\" (UniqueName: \"kubernetes.io/projected/94d461e4-7e7b-4add-94c2-aa6bacf83129-kube-api-access-xwcfv\") pod \"maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8\" (UID: \"94d461e4-7e7b-4add-94c2-aa6bacf83129\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:14:40.041522 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.041485    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwcfv\" (UniqueName: \"kubernetes.io/projected/94d461e4-7e7b-4add-94c2-aa6bacf83129-kube-api-access-xwcfv\") pod \"maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8\" (UID: \"94d461e4-7e7b-4add-94c2-aa6bacf83129\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:14:40.041710 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.041539    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94d461e4-7e7b-4add-94c2-aa6bacf83129-maas-api-tls\") pod \"maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8\" (UID: \"94d461e4-7e7b-4add-94c2-aa6bacf83129\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:14:40.041710 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.041568    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d461e4-7e7b-4add-94c2-aa6bacf83129-openshift-service-ca\") pod \"maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8\" (UID: \"94d461e4-7e7b-4add-94c2-aa6bacf83129\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:14:40.042116 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.042096    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d461e4-7e7b-4add-94c2-aa6bacf83129-openshift-service-ca\") pod \"maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8\" (UID: \"94d461e4-7e7b-4add-94c2-aa6bacf83129\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:14:40.044125 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.044101    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94d461e4-7e7b-4add-94c2-aa6bacf83129-maas-api-tls\") pod \"maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8\" (UID: \"94d461e4-7e7b-4add-94c2-aa6bacf83129\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:14:40.049930 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.049905    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwcfv\" (UniqueName: \"kubernetes.io/projected/94d461e4-7e7b-4add-94c2-aa6bacf83129-kube-api-access-xwcfv\") pod \"maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8\" (UID: \"94d461e4-7e7b-4add-94c2-aa6bacf83129\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:14:40.200164 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.200075    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:14:40.337802 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.337778    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"]
Jul 24 19:14:40.339730 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:14:40.339702    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94d461e4_7e7b_4add_94c2_aa6bacf83129.slice/crio-8b2bf3ab1981d64c3d79ee27f2ff8f9f11647c4c8c260488d5da59e41f09ab28 WatchSource:0}: Error finding container 8b2bf3ab1981d64c3d79ee27f2ff8f9f11647c4c8c260488d5da59e41f09ab28: Status 404 returned error can't find the container with id 8b2bf3ab1981d64c3d79ee27f2ff8f9f11647c4c8c260488d5da59e41f09ab28
Jul 24 19:14:40.592836 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.592797    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"]
Jul 24 19:14:40.597262 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.597240    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"
Jul 24 19:14:40.599454 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.599436    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-16b0ae71\""
Jul 24 19:14:40.604562 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.604540    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"]
Jul 24 19:14:40.619301 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.617107    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"]
Jul 24 19:14:40.621572 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.621546    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"
Jul 24 19:14:40.624273 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.624256    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-16b0ae71-dockercfg-jbj8x\""
Jul 24 19:14:40.632938 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.632910    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"]
Jul 24 19:14:40.748771 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.748737    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8f5dd73-9b66-46d1-a11d-6d3085281bd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn\" (UID: \"f8f5dd73-9b66-46d1-a11d-6d3085281bd6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"
Jul 24 19:14:40.748921 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.748794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fn65\" (UniqueName: \"kubernetes.io/projected/baeeabcf-159e-4a6b-945d-70315f5a336c-kube-api-access-2fn65\") pod \"payload-processing-e2e-mt-16b0ae71-5476657d-dc89x\" (UID: \"baeeabcf-159e-4a6b-945d-70315f5a336c\") " pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"
Jul 24 19:14:40.748921 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.748815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baeeabcf-159e-4a6b-945d-70315f5a336c-plugins-config-volume\") pod \"payload-processing-e2e-mt-16b0ae71-5476657d-dc89x\" (UID: \"baeeabcf-159e-4a6b-945d-70315f5a336c\") " pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"
Jul 24 19:14:40.748921 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.748895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnh7l\" (UniqueName: \"kubernetes.io/projected/f8f5dd73-9b66-46d1-a11d-6d3085281bd6-kube-api-access-pnh7l\") pod \"payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn\" (UID: \"f8f5dd73-9b66-46d1-a11d-6d3085281bd6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"
Jul 24 19:14:40.793779 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.793742    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8" event={"ID":"94d461e4-7e7b-4add-94c2-aa6bacf83129","Type":"ContainerStarted","Data":"8b2bf3ab1981d64c3d79ee27f2ff8f9f11647c4c8c260488d5da59e41f09ab28"}
Jul 24 19:14:40.850485 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.850421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8f5dd73-9b66-46d1-a11d-6d3085281bd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn\" (UID: \"f8f5dd73-9b66-46d1-a11d-6d3085281bd6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"
Jul 24 19:14:40.850610 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.850517    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fn65\" (UniqueName: \"kubernetes.io/projected/baeeabcf-159e-4a6b-945d-70315f5a336c-kube-api-access-2fn65\") pod \"payload-processing-e2e-mt-16b0ae71-5476657d-dc89x\" (UID: \"baeeabcf-159e-4a6b-945d-70315f5a336c\") " pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"
Jul 24 19:14:40.850610 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.850595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baeeabcf-159e-4a6b-945d-70315f5a336c-plugins-config-volume\") pod \"payload-processing-e2e-mt-16b0ae71-5476657d-dc89x\" (UID: \"baeeabcf-159e-4a6b-945d-70315f5a336c\") " pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"
Jul 24 19:14:40.850721 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.850670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnh7l\" (UniqueName: \"kubernetes.io/projected/f8f5dd73-9b66-46d1-a11d-6d3085281bd6-kube-api-access-pnh7l\") pod \"payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn\" (UID: \"f8f5dd73-9b66-46d1-a11d-6d3085281bd6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"
Jul 24 19:14:40.851215 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.851196    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8f5dd73-9b66-46d1-a11d-6d3085281bd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn\" (UID: \"f8f5dd73-9b66-46d1-a11d-6d3085281bd6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"
Jul 24 19:14:40.851303 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.851282    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baeeabcf-159e-4a6b-945d-70315f5a336c-plugins-config-volume\") pod \"payload-processing-e2e-mt-16b0ae71-5476657d-dc89x\" (UID: \"baeeabcf-159e-4a6b-945d-70315f5a336c\") " pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"
Jul 24 19:14:40.858034 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.858013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fn65\" (UniqueName: \"kubernetes.io/projected/baeeabcf-159e-4a6b-945d-70315f5a336c-kube-api-access-2fn65\") pod \"payload-processing-e2e-mt-16b0ae71-5476657d-dc89x\" (UID: \"baeeabcf-159e-4a6b-945d-70315f5a336c\") " pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"
Jul 24 19:14:40.858123 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.858014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnh7l\" (UniqueName: \"kubernetes.io/projected/f8f5dd73-9b66-46d1-a11d-6d3085281bd6-kube-api-access-pnh7l\") pod \"payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn\" (UID: \"f8f5dd73-9b66-46d1-a11d-6d3085281bd6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"
Jul 24 19:14:40.908834 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.908808    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"
Jul 24 19:14:40.932869 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:40.932844    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"
Jul 24 19:14:41.060908 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:41.060832    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"]
Jul 24 19:14:41.062251 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:14:41.062224    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f5dd73_9b66_46d1_a11d_6d3085281bd6.slice/crio-21774cd8b0088ed11ac996e1c9e01038eac1641b19139deb3d1c23ef7a3b7239 WatchSource:0}: Error finding container 21774cd8b0088ed11ac996e1c9e01038eac1641b19139deb3d1c23ef7a3b7239: Status 404 returned error can't find the container with id 21774cd8b0088ed11ac996e1c9e01038eac1641b19139deb3d1c23ef7a3b7239
Jul 24 19:14:41.089122 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:41.089075    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"]
Jul 24 19:14:41.102548 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:14:41.102516    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaeeabcf_159e_4a6b_945d_70315f5a336c.slice/crio-6872681ca50008985e952a86ca48d30bbf86b1f9c782c7e8cd8fccfb1a99230c WatchSource:0}: Error finding container 6872681ca50008985e952a86ca48d30bbf86b1f9c782c7e8cd8fccfb1a99230c: Status 404 returned error can't find the container with id 6872681ca50008985e952a86ca48d30bbf86b1f9c782c7e8cd8fccfb1a99230c
Jul 24 19:14:41.800690 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:41.800649    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x" event={"ID":"baeeabcf-159e-4a6b-945d-70315f5a336c","Type":"ContainerStarted","Data":"2d0b9c2c2870b47a1604b99a10c80567302fd22eef957aed20b9f6f8927ff38b"}
Jul 24 19:14:41.800895 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:41.800704    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x" event={"ID":"baeeabcf-159e-4a6b-945d-70315f5a336c","Type":"ContainerStarted","Data":"6872681ca50008985e952a86ca48d30bbf86b1f9c782c7e8cd8fccfb1a99230c"}
Jul 24 19:14:41.800895 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:41.800749    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"
Jul 24 19:14:41.802124 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:41.802099    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8" event={"ID":"94d461e4-7e7b-4add-94c2-aa6bacf83129","Type":"ContainerStarted","Data":"b152d022ae01daafdb825170924e7b77076967f96c577cb3f0a84c7da440429f"}
Jul 24 19:14:41.802245 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:41.802153    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:14:41.803538 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:41.803517    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn" event={"ID":"f8f5dd73-9b66-46d1-a11d-6d3085281bd6","Type":"ContainerStarted","Data":"72170385b77b7c9c3834f735e725ecde68f572a5eb0ecf2a7d8b619b71e671f3"}
Jul 24 19:14:41.803654 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:41.803552    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn" event={"ID":"f8f5dd73-9b66-46d1-a11d-6d3085281bd6","Type":"ContainerStarted","Data":"21774cd8b0088ed11ac996e1c9e01038eac1641b19139deb3d1c23ef7a3b7239"}
Jul 24 19:14:41.803654 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:41.803596    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"
Jul 24 19:14:41.821725 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:41.821672    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x" podStartSLOduration=1.8216559270000001 podStartE2EDuration="1.821655927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:14:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:14:41.818906218 +0000 UTC m=+1820.209857699" watchObservedRunningTime="2026-07-24 19:14:41.821655927 +0000 UTC m=+1820.212607403"
Jul 24 19:14:41.835723 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:41.835678    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8" podStartSLOduration=2.351062652 podStartE2EDuration="2.835665087s" totalImagesPullingTime="484.602435ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:14:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:14:41.83410744 +0000 UTC m=+1820.225058917" watchObservedRunningTime="2026-07-24 19:14:41.835665087 +0000 UTC m=+1820.226616564"
Jul 24 19:14:41.850069 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:41.850030    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn" podStartSLOduration=1.850016687 podStartE2EDuration="1.850016687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:14:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:14:41.849015951 +0000 UTC m=+1820.239967428" watchObservedRunningTime="2026-07-24 19:14:41.850016687 +0000 UTC m=+1820.240968164"
Jul 24 19:14:46.545757 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:46.545722    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:14:47.823118 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:47.823086    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:14:52.810814 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:52.810781    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"
Jul 24 19:14:52.811317 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:52.810862    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"
Jul 24 19:14:54.237658 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:54.237624    2563 patch_prober.go:28] interesting pod/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:48584->10.134.0.40:15021: read: connection reset by peer" start-of-body=
Jul 24 19:14:54.238168 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:54.237740    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n" podUID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:48584->10.134.0.40:15021: read: connection reset by peer"
Jul 24 19:14:54.874597 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:54.874562    2563 generic.go:356] "Generic (PLEG): container finished" podID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" containerID="c89fd0d6543621aae642d31c91ec43b01e1b07acfa8df25306de40ac692e39d7" exitCode=0
Jul 24 19:14:54.874793 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:54.874633    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n" event={"ID":"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3","Type":"ContainerDied","Data":"c89fd0d6543621aae642d31c91ec43b01e1b07acfa8df25306de40ac692e39d7"}
Jul 24 19:14:54.875050 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:54.875034    2563 scope.go:117] "RemoveContainer" containerID="c89fd0d6543621aae642d31c91ec43b01e1b07acfa8df25306de40ac692e39d7"
Jul 24 19:14:54.876157 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:54.876130    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:14:54.876231 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:54.876199    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:14:54.876267 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:54.876234    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:14:55.881366 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:55.881331    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n" event={"ID":"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3","Type":"ContainerStarted","Data":"af0e0c1a980875d9272d1a53fb281814e2676f1d248a088eefb7714fae624773"}
Jul 24 19:14:56.728826 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:56.728790    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:56.729219 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:56.729196    2563 patch_prober.go:28] interesting pod/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 24 19:14:56.729382 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:56.729354    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n" podUID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 24 19:14:57.729416 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:57.729345    2563 patch_prober.go:28] interesting pod/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 24 19:14:57.729926 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:57.729443    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n" podUID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 24 19:14:58.729065 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:58.729026    2563 patch_prober.go:28] interesting pod/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 24 19:14:58.729356 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:58.729329    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n" podUID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 24 19:14:59.732461 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:59.732427    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:59.732922 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:59.732676    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:14:59.733362 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:14:59.733343    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:15:01.667581 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:01.667541    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"]
Jul 24 19:15:01.668039 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:01.667773    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8" podUID="94d461e4-7e7b-4add-94c2-aa6bacf83129" containerName="maas-api" containerID="cri-o://b152d022ae01daafdb825170924e7b77076967f96c577cb3f0a84c7da440429f" gracePeriod=30
Jul 24 19:15:01.740957 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:01.740915    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"]
Jul 24 19:15:01.741216 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:01.741162    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x" podUID="baeeabcf-159e-4a6b-945d-70315f5a336c" containerName="payload-processing" containerID="cri-o://2d0b9c2c2870b47a1604b99a10c80567302fd22eef957aed20b9f6f8927ff38b" gracePeriod=30
Jul 24 19:15:01.748722 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:01.748681    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"]
Jul 24 19:15:01.748933 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:01.748910    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn" podUID="f8f5dd73-9b66-46d1-a11d-6d3085281bd6" containerName="payload-pre-processing" containerID="cri-o://72170385b77b7c9c3834f735e725ecde68f572a5eb0ecf2a7d8b619b71e671f3" gracePeriod=30
Jul 24 19:15:01.917778 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:01.915831    2563 generic.go:356] "Generic (PLEG): container finished" podID="baeeabcf-159e-4a6b-945d-70315f5a336c" containerID="2d0b9c2c2870b47a1604b99a10c80567302fd22eef957aed20b9f6f8927ff38b" exitCode=0
Jul 24 19:15:01.917778 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:01.915941    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x" event={"ID":"baeeabcf-159e-4a6b-945d-70315f5a336c","Type":"ContainerDied","Data":"2d0b9c2c2870b47a1604b99a10c80567302fd22eef957aed20b9f6f8927ff38b"}
Jul 24 19:15:01.918342 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:01.918314    2563 generic.go:356] "Generic (PLEG): container finished" podID="94d461e4-7e7b-4add-94c2-aa6bacf83129" containerID="b152d022ae01daafdb825170924e7b77076967f96c577cb3f0a84c7da440429f" exitCode=0
Jul 24 19:15:01.918876 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:01.918785    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8" event={"ID":"94d461e4-7e7b-4add-94c2-aa6bacf83129","Type":"ContainerDied","Data":"b152d022ae01daafdb825170924e7b77076967f96c577cb3f0a84c7da440429f"}
Jul 24 19:15:01.922219 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:01.922192    2563 generic.go:356] "Generic (PLEG): container finished" podID="f8f5dd73-9b66-46d1-a11d-6d3085281bd6" containerID="72170385b77b7c9c3834f735e725ecde68f572a5eb0ecf2a7d8b619b71e671f3" exitCode=0
Jul 24 19:15:01.922331 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:01.922284    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn" event={"ID":"f8f5dd73-9b66-46d1-a11d-6d3085281bd6","Type":"ContainerDied","Data":"72170385b77b7c9c3834f735e725ecde68f572a5eb0ecf2a7d8b619b71e671f3"}
Jul 24 19:15:01.999070 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:01.999006    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:15:02.038575 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.038542    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"
Jul 24 19:15:02.041575 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.041552    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94d461e4-7e7b-4add-94c2-aa6bacf83129-maas-api-tls\") pod \"94d461e4-7e7b-4add-94c2-aa6bacf83129\" (UID: \"94d461e4-7e7b-4add-94c2-aa6bacf83129\") "
Jul 24 19:15:02.041703 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.041640    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwcfv\" (UniqueName: \"kubernetes.io/projected/94d461e4-7e7b-4add-94c2-aa6bacf83129-kube-api-access-xwcfv\") pod \"94d461e4-7e7b-4add-94c2-aa6bacf83129\" (UID: \"94d461e4-7e7b-4add-94c2-aa6bacf83129\") "
Jul 24 19:15:02.041703 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.041664    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d461e4-7e7b-4add-94c2-aa6bacf83129-openshift-service-ca\") pod \"94d461e4-7e7b-4add-94c2-aa6bacf83129\" (UID: \"94d461e4-7e7b-4add-94c2-aa6bacf83129\") "
Jul 24 19:15:02.041703 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.041695    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baeeabcf-159e-4a6b-945d-70315f5a336c-plugins-config-volume\") pod \"baeeabcf-159e-4a6b-945d-70315f5a336c\" (UID: \"baeeabcf-159e-4a6b-945d-70315f5a336c\") "
Jul 24 19:15:02.041888 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.041718    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fn65\" (UniqueName: \"kubernetes.io/projected/baeeabcf-159e-4a6b-945d-70315f5a336c-kube-api-access-2fn65\") pod \"baeeabcf-159e-4a6b-945d-70315f5a336c\" (UID: \"baeeabcf-159e-4a6b-945d-70315f5a336c\") "
Jul 24 19:15:02.042154 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.042103    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94d461e4-7e7b-4add-94c2-aa6bacf83129-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "94d461e4-7e7b-4add-94c2-aa6bacf83129" (UID: "94d461e4-7e7b-4add-94c2-aa6bacf83129"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:15:02.042249 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.042157    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baeeabcf-159e-4a6b-945d-70315f5a336c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "baeeabcf-159e-4a6b-945d-70315f5a336c" (UID: "baeeabcf-159e-4a6b-945d-70315f5a336c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:15:02.043818 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.043788    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94d461e4-7e7b-4add-94c2-aa6bacf83129-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "94d461e4-7e7b-4add-94c2-aa6bacf83129" (UID: "94d461e4-7e7b-4add-94c2-aa6bacf83129"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:15:02.044148 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.044130    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baeeabcf-159e-4a6b-945d-70315f5a336c-kube-api-access-2fn65" (OuterVolumeSpecName: "kube-api-access-2fn65") pod "baeeabcf-159e-4a6b-945d-70315f5a336c" (UID: "baeeabcf-159e-4a6b-945d-70315f5a336c"). InnerVolumeSpecName "kube-api-access-2fn65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:15:02.044264 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.044251    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94d461e4-7e7b-4add-94c2-aa6bacf83129-kube-api-access-xwcfv" (OuterVolumeSpecName: "kube-api-access-xwcfv") pod "94d461e4-7e7b-4add-94c2-aa6bacf83129" (UID: "94d461e4-7e7b-4add-94c2-aa6bacf83129"). InnerVolumeSpecName "kube-api-access-xwcfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:15:02.048524 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.048505    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"
Jul 24 19:15:02.142181 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.142139    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8f5dd73-9b66-46d1-a11d-6d3085281bd6-plugins-config-volume\") pod \"f8f5dd73-9b66-46d1-a11d-6d3085281bd6\" (UID: \"f8f5dd73-9b66-46d1-a11d-6d3085281bd6\") "
Jul 24 19:15:02.142359 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.142240    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnh7l\" (UniqueName: \"kubernetes.io/projected/f8f5dd73-9b66-46d1-a11d-6d3085281bd6-kube-api-access-pnh7l\") pod \"f8f5dd73-9b66-46d1-a11d-6d3085281bd6\" (UID: \"f8f5dd73-9b66-46d1-a11d-6d3085281bd6\") "
Jul 24 19:15:02.142436 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.142369    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94d461e4-7e7b-4add-94c2-aa6bacf83129-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:02.142436 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.142380    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwcfv\" (UniqueName: \"kubernetes.io/projected/94d461e4-7e7b-4add-94c2-aa6bacf83129-kube-api-access-xwcfv\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:02.142436 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.142421    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d461e4-7e7b-4add-94c2-aa6bacf83129-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:02.142436 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.142430    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baeeabcf-159e-4a6b-945d-70315f5a336c-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:02.142562 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.142439    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fn65\" (UniqueName: \"kubernetes.io/projected/baeeabcf-159e-4a6b-945d-70315f5a336c-kube-api-access-2fn65\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:02.142679 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.142648    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8f5dd73-9b66-46d1-a11d-6d3085281bd6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f8f5dd73-9b66-46d1-a11d-6d3085281bd6" (UID: "f8f5dd73-9b66-46d1-a11d-6d3085281bd6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:15:02.144558 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.144526    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8f5dd73-9b66-46d1-a11d-6d3085281bd6-kube-api-access-pnh7l" (OuterVolumeSpecName: "kube-api-access-pnh7l") pod "f8f5dd73-9b66-46d1-a11d-6d3085281bd6" (UID: "f8f5dd73-9b66-46d1-a11d-6d3085281bd6"). InnerVolumeSpecName "kube-api-access-pnh7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:15:02.243889 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.243854    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnh7l\" (UniqueName: \"kubernetes.io/projected/f8f5dd73-9b66-46d1-a11d-6d3085281bd6-kube-api-access-pnh7l\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:02.243889 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.243887    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8f5dd73-9b66-46d1-a11d-6d3085281bd6-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:02.928064 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.928033    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"
Jul 24 19:15:02.928064 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.928051    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn" event={"ID":"f8f5dd73-9b66-46d1-a11d-6d3085281bd6","Type":"ContainerDied","Data":"21774cd8b0088ed11ac996e1c9e01038eac1641b19139deb3d1c23ef7a3b7239"}
Jul 24 19:15:02.928611 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.928138    2563 scope.go:117] "RemoveContainer" containerID="72170385b77b7c9c3834f735e725ecde68f572a5eb0ecf2a7d8b619b71e671f3"
Jul 24 19:15:02.929537 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.929522    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"
Jul 24 19:15:02.929615 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.929534    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x" event={"ID":"baeeabcf-159e-4a6b-945d-70315f5a336c","Type":"ContainerDied","Data":"6872681ca50008985e952a86ca48d30bbf86b1f9c782c7e8cd8fccfb1a99230c"}
Jul 24 19:15:02.931566 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.931492    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8" event={"ID":"94d461e4-7e7b-4add-94c2-aa6bacf83129","Type":"ContainerDied","Data":"8b2bf3ab1981d64c3d79ee27f2ff8f9f11647c4c8c260488d5da59e41f09ab28"}
Jul 24 19:15:02.931566 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.931515    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"
Jul 24 19:15:02.938110 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.938095    2563 scope.go:117] "RemoveContainer" containerID="2d0b9c2c2870b47a1604b99a10c80567302fd22eef957aed20b9f6f8927ff38b"
Jul 24 19:15:02.946215 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.946192    2563 scope.go:117] "RemoveContainer" containerID="b152d022ae01daafdb825170924e7b77076967f96c577cb3f0a84c7da440429f"
Jul 24 19:15:02.949845 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.949819    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"]
Jul 24 19:15:02.955122 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.955093    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-16b0ae71-58b5f48c4f-rwgm8"]
Jul 24 19:15:02.963606 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.963581    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"]
Jul 24 19:15:02.967578 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.967553    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-16b0ae71-7cbcc485fb-c6rjn"]
Jul 24 19:15:02.978170 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.978145    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"]
Jul 24 19:15:02.981548 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:02.981525    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-16b0ae71-5476657d-dc89x"]
Jul 24 19:15:04.203508 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:04.203475    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94d461e4-7e7b-4add-94c2-aa6bacf83129" path="/var/lib/kubelet/pods/94d461e4-7e7b-4add-94c2-aa6bacf83129/volumes"
Jul 24 19:15:04.203898 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:04.203822    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baeeabcf-159e-4a6b-945d-70315f5a336c" path="/var/lib/kubelet/pods/baeeabcf-159e-4a6b-945d-70315f5a336c/volumes"
Jul 24 19:15:04.204117 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:04.204105    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8f5dd73-9b66-46d1-a11d-6d3085281bd6" path="/var/lib/kubelet/pods/f8f5dd73-9b66-46d1-a11d-6d3085281bd6/volumes"
Jul 24 19:15:04.658271 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:04.658227    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:15:13.135513 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:13.135484    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"]
Jul 24 19:15:13.135956 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:13.135731    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n" podUID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" containerName="istio-proxy" containerID="cri-o://af0e0c1a980875d9272d1a53fb281814e2676f1d248a088eefb7714fae624773" gracePeriod=30
Jul 24 19:15:14.735052 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:14.735006    2563 patch_prober.go:28] interesting pod/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 19:15:14.735461 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:14.735072    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n" podUID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 19:15:18.389332 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.389308    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:15:18.478812 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.478737    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-credential-socket\") pod \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") "
Jul 24 19:15:18.478812 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.478775    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-workload-socket\") pod \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") "
Jul 24 19:15:18.478812 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.478795    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-token\") pod \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") "
Jul 24 19:15:18.478812 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.478809    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-envoy\") pod \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") "
Jul 24 19:15:18.479100 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.478849    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-data\") pod \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") "
Jul 24 19:15:18.479100 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.478873    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-workload-certs\") pod \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") "
Jul 24 19:15:18.479100 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.478889    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istiod-ca-cert\") pod \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") "
Jul 24 19:15:18.479100 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.478925    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t5sd\" (UniqueName: \"kubernetes.io/projected/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-kube-api-access-9t5sd\") pod \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") "
Jul 24 19:15:18.479100 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.478960    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-podinfo\") pod \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\" (UID: \"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3\") "
Jul 24 19:15:18.479100 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.479012    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" (UID: "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:15:18.479100 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.479086    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" (UID: "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:15:18.479439 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.479247    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-credential-socket\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:18.479439 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.479269    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-workload-socket\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:18.479439 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.479268    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" (UID: "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:15:18.479439 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.479352    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" (UID: "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:15:18.480181 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.480151    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" (UID: "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:15:18.481175 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.481147    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" (UID: "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:15:18.481175 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.481157    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" (UID: "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:15:18.481340 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.481253    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-kube-api-access-9t5sd" (OuterVolumeSpecName: "kube-api-access-9t5sd") pod "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" (UID: "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3"). InnerVolumeSpecName "kube-api-access-9t5sd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:15:18.481645 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.481631    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" (UID: "ba66457d-d0d8-4c2c-9cb4-d98bc15308f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 19:15:18.579950 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.579919    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-token\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:18.579950 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.579946    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-envoy\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:18.579950 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.579954    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-data\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:18.580166 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.579962    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-workload-certs\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:18.580166 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.579970    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istiod-ca-cert\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:18.580166 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.579977    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t5sd\" (UniqueName: \"kubernetes.io/projected/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-kube-api-access-9t5sd\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:18.580166 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:18.579987    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3-istio-podinfo\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:15:19.012851 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:19.012812    2563 generic.go:356] "Generic (PLEG): container finished" podID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" containerID="af0e0c1a980875d9272d1a53fb281814e2676f1d248a088eefb7714fae624773" exitCode=0
Jul 24 19:15:19.013039 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:19.012862    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"
Jul 24 19:15:19.013039 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:19.012896    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n" event={"ID":"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3","Type":"ContainerDied","Data":"af0e0c1a980875d9272d1a53fb281814e2676f1d248a088eefb7714fae624773"}
Jul 24 19:15:19.013039 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:19.012957    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n" event={"ID":"ba66457d-d0d8-4c2c-9cb4-d98bc15308f3","Type":"ContainerDied","Data":"9ba22db73b0cadbb79ff09de8a33a672267d2f0bd7704f8d2863766569298517"}
Jul 24 19:15:19.013039 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:19.012988    2563 scope.go:117] "RemoveContainer" containerID="af0e0c1a980875d9272d1a53fb281814e2676f1d248a088eefb7714fae624773"
Jul 24 19:15:19.023858 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:19.023832    2563 scope.go:117] "RemoveContainer" containerID="c89fd0d6543621aae642d31c91ec43b01e1b07acfa8df25306de40ac692e39d7"
Jul 24 19:15:19.032197 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:19.032177    2563 scope.go:117] "RemoveContainer" containerID="af0e0c1a980875d9272d1a53fb281814e2676f1d248a088eefb7714fae624773"
Jul 24 19:15:19.032465 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:15:19.032444    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af0e0c1a980875d9272d1a53fb281814e2676f1d248a088eefb7714fae624773\": container with ID starting with af0e0c1a980875d9272d1a53fb281814e2676f1d248a088eefb7714fae624773 not found: ID does not exist" containerID="af0e0c1a980875d9272d1a53fb281814e2676f1d248a088eefb7714fae624773"
Jul 24 19:15:19.032561 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:19.032478    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af0e0c1a980875d9272d1a53fb281814e2676f1d248a088eefb7714fae624773"} err="failed to get container status \"af0e0c1a980875d9272d1a53fb281814e2676f1d248a088eefb7714fae624773\": rpc error: code = NotFound desc = could not find container \"af0e0c1a980875d9272d1a53fb281814e2676f1d248a088eefb7714fae624773\": container with ID starting with af0e0c1a980875d9272d1a53fb281814e2676f1d248a088eefb7714fae624773 not found: ID does not exist"
Jul 24 19:15:19.032561 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:19.032503    2563 scope.go:117] "RemoveContainer" containerID="c89fd0d6543621aae642d31c91ec43b01e1b07acfa8df25306de40ac692e39d7"
Jul 24 19:15:19.032772 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:15:19.032755    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c89fd0d6543621aae642d31c91ec43b01e1b07acfa8df25306de40ac692e39d7\": container with ID starting with c89fd0d6543621aae642d31c91ec43b01e1b07acfa8df25306de40ac692e39d7 not found: ID does not exist" containerID="c89fd0d6543621aae642d31c91ec43b01e1b07acfa8df25306de40ac692e39d7"
Jul 24 19:15:19.032819 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:19.032782    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c89fd0d6543621aae642d31c91ec43b01e1b07acfa8df25306de40ac692e39d7"} err="failed to get container status \"c89fd0d6543621aae642d31c91ec43b01e1b07acfa8df25306de40ac692e39d7\": rpc error: code = NotFound desc = could not find container \"c89fd0d6543621aae642d31c91ec43b01e1b07acfa8df25306de40ac692e39d7\": container with ID starting with c89fd0d6543621aae642d31c91ec43b01e1b07acfa8df25306de40ac692e39d7 not found: ID does not exist"
Jul 24 19:15:19.041579 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:19.041552    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"]
Jul 24 19:15:19.043343 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:19.043325    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-16b0ae71-openshift-default-54fc8f7fd8-mxc9n"]
Jul 24 19:15:20.203131 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:20.203100    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" path="/var/lib/kubelet/pods/ba66457d-d0d8-4c2c-9cb4-d98bc15308f3/volumes"
Jul 24 19:15:32.839959 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.839919    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"]
Jul 24 19:15:32.840457 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840351    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94d461e4-7e7b-4add-94c2-aa6bacf83129" containerName="maas-api"
Jul 24 19:15:32.840457 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840369    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="94d461e4-7e7b-4add-94c2-aa6bacf83129" containerName="maas-api"
Jul 24 19:15:32.840457 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840384    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8f5dd73-9b66-46d1-a11d-6d3085281bd6" containerName="payload-pre-processing"
Jul 24 19:15:32.840457 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840411    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8f5dd73-9b66-46d1-a11d-6d3085281bd6" containerName="payload-pre-processing"
Jul 24 19:15:32.840457 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840420    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baeeabcf-159e-4a6b-945d-70315f5a336c" containerName="payload-processing"
Jul 24 19:15:32.840457 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840428    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="baeeabcf-159e-4a6b-945d-70315f5a336c" containerName="payload-processing"
Jul 24 19:15:32.840457 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840460    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" containerName="istio-proxy"
Jul 24 19:15:32.840858 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840468    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" containerName="istio-proxy"
Jul 24 19:15:32.840858 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840483    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" containerName="istio-proxy"
Jul 24 19:15:32.840858 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840491    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" containerName="istio-proxy"
Jul 24 19:15:32.840858 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840571    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="baeeabcf-159e-4a6b-945d-70315f5a336c" containerName="payload-processing"
Jul 24 19:15:32.840858 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840586    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" containerName="istio-proxy"
Jul 24 19:15:32.840858 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840595    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba66457d-d0d8-4c2c-9cb4-d98bc15308f3" containerName="istio-proxy"
Jul 24 19:15:32.840858 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840607    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8f5dd73-9b66-46d1-a11d-6d3085281bd6" containerName="payload-pre-processing"
Jul 24 19:15:32.840858 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.840617    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="94d461e4-7e7b-4add-94c2-aa6bacf83129" containerName="maas-api"
Jul 24 19:15:32.843504 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.843485    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:32.845922 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.845901    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7baf010d-openshift-default-dockercfg-wpf7r\""
Jul 24 19:15:32.857002 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.856970    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"]
Jul 24 19:15:32.900132 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.900099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-credential-socket\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:32.900132 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.900137    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-workload-socket\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:32.900353 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.900175    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-workload-certs\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:32.900353 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.900251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjcc5\" (UniqueName: \"kubernetes.io/projected/a513b468-4856-4c3f-b22b-061ffd77bd5e-kube-api-access-sjcc5\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:32.900353 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.900292    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a513b468-4856-4c3f-b22b-061ffd77bd5e-istiod-ca-cert\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:32.900353 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.900321    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-envoy\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:32.900353 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.900337    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-data\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:32.900353 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.900351    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-podinfo\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:32.900604 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:32.900365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-token\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.000805 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.000763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjcc5\" (UniqueName: \"kubernetes.io/projected/a513b468-4856-4c3f-b22b-061ffd77bd5e-kube-api-access-sjcc5\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.001009 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.000816    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a513b468-4856-4c3f-b22b-061ffd77bd5e-istiod-ca-cert\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.001009 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.000930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-envoy\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.001009 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.000955    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-data\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.001009 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.000974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-podinfo\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.001009 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.000989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-token\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.001298 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.001024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-credential-socket\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.001298 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.001057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-workload-socket\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.001298 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.001128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-workload-certs\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.001497 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.001429    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-data\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.001595 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.001573    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-workload-socket\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.001651 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.001601    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-workload-certs\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.001690 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.001665    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-credential-socket\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.001728 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.001686    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a513b468-4856-4c3f-b22b-061ffd77bd5e-istiod-ca-cert\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.003637 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.003619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-envoy\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.003758 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.003742    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-podinfo\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.008690 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.008660    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-token\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.008938 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.008919    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjcc5\" (UniqueName: \"kubernetes.io/projected/a513b468-4856-4c3f-b22b-061ffd77bd5e-kube-api-access-sjcc5\") pod \"e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") " pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.154515 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.154426    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:33.286298 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.286270    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"]
Jul 24 19:15:33.288479 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:15:33.288453    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda513b468_4856_4c3f_b22b_061ffd77bd5e.slice/crio-c7e65fdfdbf2c05cf848f02705b236d19a9e278cc2d9bcbad70a4b786604ec91 WatchSource:0}: Error finding container c7e65fdfdbf2c05cf848f02705b236d19a9e278cc2d9bcbad70a4b786604ec91: Status 404 returned error can't find the container with id c7e65fdfdbf2c05cf848f02705b236d19a9e278cc2d9bcbad70a4b786604ec91
Jul 24 19:15:33.290932 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.290899    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:15:33.291047 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.290964    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:15:33.291047 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:33.290989    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:15:34.083908 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:34.083872    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr" event={"ID":"a513b468-4856-4c3f-b22b-061ffd77bd5e","Type":"ContainerStarted","Data":"81eba83cc1e39ba94024a04a0f4f1ecfc2a7912ed97d3d5c99225f465770d092"}
Jul 24 19:15:34.084301 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:34.083927    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr" event={"ID":"a513b468-4856-4c3f-b22b-061ffd77bd5e","Type":"ContainerStarted","Data":"c7e65fdfdbf2c05cf848f02705b236d19a9e278cc2d9bcbad70a4b786604ec91"}
Jul 24 19:15:34.104985 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:34.104939    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr" podStartSLOduration=2.104924214 podStartE2EDuration="2.104924214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:15:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:15:34.103290006 +0000 UTC m=+1872.494241508" watchObservedRunningTime="2026-07-24 19:15:34.104924214 +0000 UTC m=+1872.495875691"
Jul 24 19:15:34.155090 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:34.155059    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:34.160225 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:34.160193    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:35.088088 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:35.088051    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:35.088969 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:35.088951    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:36.090984 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.090950    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"]
Jul 24 19:15:36.093269 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.093249    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:15:36.095777 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.095752    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7baf010d-serving-cert\""
Jul 24 19:15:36.095897 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.095776    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 19:15:36.095897 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.095776    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cvjqn\""
Jul 24 19:15:36.096620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.096598    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 19:15:36.104673 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.104651    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"]
Jul 24 19:15:36.229203 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.229172    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b989043f-ca9c-4071-a75d-e648724c3b46-maas-api-tls\") pod \"maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts\" (UID: \"b989043f-ca9c-4071-a75d-e648724c3b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:15:36.229433 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.229212    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wj6c\" (UniqueName: \"kubernetes.io/projected/b989043f-ca9c-4071-a75d-e648724c3b46-kube-api-access-6wj6c\") pod \"maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts\" (UID: \"b989043f-ca9c-4071-a75d-e648724c3b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:15:36.229433 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.229276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b989043f-ca9c-4071-a75d-e648724c3b46-openshift-service-ca\") pod \"maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts\" (UID: \"b989043f-ca9c-4071-a75d-e648724c3b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:15:36.330694 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.330664    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b989043f-ca9c-4071-a75d-e648724c3b46-maas-api-tls\") pod \"maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts\" (UID: \"b989043f-ca9c-4071-a75d-e648724c3b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:15:36.330892 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.330719    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wj6c\" (UniqueName: \"kubernetes.io/projected/b989043f-ca9c-4071-a75d-e648724c3b46-kube-api-access-6wj6c\") pod \"maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts\" (UID: \"b989043f-ca9c-4071-a75d-e648724c3b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:15:36.330892 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:15:36.330818    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-serving-cert: secret "maas-api-e2e-mt-7baf010d-serving-cert" not found
Jul 24 19:15:36.330892 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:15:36.330878    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b989043f-ca9c-4071-a75d-e648724c3b46-maas-api-tls podName:b989043f-ca9c-4071-a75d-e648724c3b46 nodeName:}" failed. No retries permitted until 2026-07-24 19:15:36.830861141 +0000 UTC m=+1875.221812599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b989043f-ca9c-4071-a75d-e648724c3b46-maas-api-tls") pod "maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts" (UID: "b989043f-ca9c-4071-a75d-e648724c3b46") : secret "maas-api-e2e-mt-7baf010d-serving-cert" not found
Jul 24 19:15:36.330892 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.330873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b989043f-ca9c-4071-a75d-e648724c3b46-openshift-service-ca\") pod \"maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts\" (UID: \"b989043f-ca9c-4071-a75d-e648724c3b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:15:36.331547 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.331527    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b989043f-ca9c-4071-a75d-e648724c3b46-openshift-service-ca\") pod \"maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts\" (UID: \"b989043f-ca9c-4071-a75d-e648724c3b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:15:36.348273 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.348216    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wj6c\" (UniqueName: \"kubernetes.io/projected/b989043f-ca9c-4071-a75d-e648724c3b46-kube-api-access-6wj6c\") pod \"maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts\" (UID: \"b989043f-ca9c-4071-a75d-e648724c3b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:15:36.813485 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.813437    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"]
Jul 24 19:15:36.816087 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.816064    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"
Jul 24 19:15:36.818754 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.818722    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-7baf010d-dockercfg-9qc7g\""
Jul 24 19:15:36.818911 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.818891    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-7baf010d\""
Jul 24 19:15:36.828558 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.828536    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"]
Jul 24 19:15:36.835266 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.835241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b989043f-ca9c-4071-a75d-e648724c3b46-maas-api-tls\") pod \"maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts\" (UID: \"b989043f-ca9c-4071-a75d-e648724c3b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:15:36.837235 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.837214    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"]
Jul 24 19:15:36.837938 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.837911    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b989043f-ca9c-4071-a75d-e648724c3b46-maas-api-tls\") pod \"maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts\" (UID: \"b989043f-ca9c-4071-a75d-e648724c3b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:15:36.839628 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.839605    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"
Jul 24 19:15:36.853442 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.853420    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"]
Jul 24 19:15:36.936378 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.936347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6704c83-513e-477e-99e6-5973ace3205a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7\" (UID: \"b6704c83-513e-477e-99e6-5973ace3205a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"
Jul 24 19:15:36.936587 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.936412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfm8s\" (UniqueName: \"kubernetes.io/projected/b6704c83-513e-477e-99e6-5973ace3205a-kube-api-access-sfm8s\") pod \"payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7\" (UID: \"b6704c83-513e-477e-99e6-5973ace3205a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"
Jul 24 19:15:36.936587 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.936488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghqsg\" (UniqueName: \"kubernetes.io/projected/9f68222d-e1d8-45ae-977f-033649f1938b-kube-api-access-ghqsg\") pod \"payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9\" (UID: \"9f68222d-e1d8-45ae-977f-033649f1938b\") " pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"
Jul 24 19:15:36.936587 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:36.936528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f68222d-e1d8-45ae-977f-033649f1938b-plugins-config-volume\") pod \"payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9\" (UID: \"9f68222d-e1d8-45ae-977f-033649f1938b\") " pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"
Jul 24 19:15:37.004557 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.004523    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:15:37.037695 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.037661    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6704c83-513e-477e-99e6-5973ace3205a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7\" (UID: \"b6704c83-513e-477e-99e6-5973ace3205a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"
Jul 24 19:15:37.037855 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.037717    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfm8s\" (UniqueName: \"kubernetes.io/projected/b6704c83-513e-477e-99e6-5973ace3205a-kube-api-access-sfm8s\") pod \"payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7\" (UID: \"b6704c83-513e-477e-99e6-5973ace3205a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"
Jul 24 19:15:37.037855 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.037750    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghqsg\" (UniqueName: \"kubernetes.io/projected/9f68222d-e1d8-45ae-977f-033649f1938b-kube-api-access-ghqsg\") pod \"payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9\" (UID: \"9f68222d-e1d8-45ae-977f-033649f1938b\") " pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"
Jul 24 19:15:37.037855 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.037771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f68222d-e1d8-45ae-977f-033649f1938b-plugins-config-volume\") pod \"payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9\" (UID: \"9f68222d-e1d8-45ae-977f-033649f1938b\") " pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"
Jul 24 19:15:37.038439 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.038422    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f68222d-e1d8-45ae-977f-033649f1938b-plugins-config-volume\") pod \"payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9\" (UID: \"9f68222d-e1d8-45ae-977f-033649f1938b\") " pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"
Jul 24 19:15:37.038813 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.038790    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6704c83-513e-477e-99e6-5973ace3205a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7\" (UID: \"b6704c83-513e-477e-99e6-5973ace3205a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"
Jul 24 19:15:37.046590 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.046565    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghqsg\" (UniqueName: \"kubernetes.io/projected/9f68222d-e1d8-45ae-977f-033649f1938b-kube-api-access-ghqsg\") pod \"payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9\" (UID: \"9f68222d-e1d8-45ae-977f-033649f1938b\") " pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"
Jul 24 19:15:37.047125 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.047107    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfm8s\" (UniqueName: \"kubernetes.io/projected/b6704c83-513e-477e-99e6-5973ace3205a-kube-api-access-sfm8s\") pod \"payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7\" (UID: \"b6704c83-513e-477e-99e6-5973ace3205a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"
Jul 24 19:15:37.127287 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.127255    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"
Jul 24 19:15:37.143453 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.143424    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"]
Jul 24 19:15:37.143926 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:15:37.143901    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb989043f_ca9c_4071_a75d_e648724c3b46.slice/crio-d28461f1947e7ebde6b3dfee8344bb03680a18c3d0a46651b7595061ccaea0f7 WatchSource:0}: Error finding container d28461f1947e7ebde6b3dfee8344bb03680a18c3d0a46651b7595061ccaea0f7: Status 404 returned error can't find the container with id d28461f1947e7ebde6b3dfee8344bb03680a18c3d0a46651b7595061ccaea0f7
Jul 24 19:15:37.145318 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.145303    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 19:15:37.154316 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.154292    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"
Jul 24 19:15:37.268311 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.268277    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"]
Jul 24 19:15:37.271292 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:15:37.271257    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6704c83_513e_477e_99e6_5973ace3205a.slice/crio-859cd7d76793e75e25135b24feb44233fe8ac0ee174bda116db56dc0d5cd38dd WatchSource:0}: Error finding container 859cd7d76793e75e25135b24feb44233fe8ac0ee174bda116db56dc0d5cd38dd: Status 404 returned error can't find the container with id 859cd7d76793e75e25135b24feb44233fe8ac0ee174bda116db56dc0d5cd38dd
Jul 24 19:15:37.301362 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:37.301341    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"]
Jul 24 19:15:37.302920 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:15:37.302890    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f68222d_e1d8_45ae_977f_033649f1938b.slice/crio-eb5f22e21d251af67a8b44c1d22a4bf5207b4740859969bc7661bcff30929ae7 WatchSource:0}: Error finding container eb5f22e21d251af67a8b44c1d22a4bf5207b4740859969bc7661bcff30929ae7: Status 404 returned error can't find the container with id eb5f22e21d251af67a8b44c1d22a4bf5207b4740859969bc7661bcff30929ae7
Jul 24 19:15:38.106381 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:38.106343    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7" event={"ID":"b6704c83-513e-477e-99e6-5973ace3205a","Type":"ContainerStarted","Data":"fe917c11ff3b9c947a01df7d1542f5fb15ec2b47bdd514fbc122f2b167d73a20"}
Jul 24 19:15:38.106602 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:38.106431    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7" event={"ID":"b6704c83-513e-477e-99e6-5973ace3205a","Type":"ContainerStarted","Data":"859cd7d76793e75e25135b24feb44233fe8ac0ee174bda116db56dc0d5cd38dd"}
Jul 24 19:15:38.106602 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:38.106462    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"
Jul 24 19:15:38.107904 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:38.107877    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9" event={"ID":"9f68222d-e1d8-45ae-977f-033649f1938b","Type":"ContainerStarted","Data":"a4f683fa70df729e55cc75b2755392a4f2d87a8fece7de9afb53cfb21f21ff37"}
Jul 24 19:15:38.108015 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:38.107928    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"
Jul 24 19:15:38.108015 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:38.107952    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9" event={"ID":"9f68222d-e1d8-45ae-977f-033649f1938b","Type":"ContainerStarted","Data":"eb5f22e21d251af67a8b44c1d22a4bf5207b4740859969bc7661bcff30929ae7"}
Jul 24 19:15:38.109574 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:38.109550    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts" event={"ID":"b989043f-ca9c-4071-a75d-e648724c3b46","Type":"ContainerStarted","Data":"fd2f59b625da49195ab5464adfce5f58c0e68a38a127c37dc084022cf5d26cf0"}
Jul 24 19:15:38.109690 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:38.109595    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts" event={"ID":"b989043f-ca9c-4071-a75d-e648724c3b46","Type":"ContainerStarted","Data":"d28461f1947e7ebde6b3dfee8344bb03680a18c3d0a46651b7595061ccaea0f7"}
Jul 24 19:15:38.109690 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:38.109623    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:15:38.124291 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:38.124247    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7" podStartSLOduration=2.124231794 podStartE2EDuration="2.124231794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:15:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:15:38.12241583 +0000 UTC m=+1876.513367302" watchObservedRunningTime="2026-07-24 19:15:38.124231794 +0000 UTC m=+1876.515183249"
Jul 24 19:15:38.142213 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:38.142171    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts" podStartSLOduration=1.664144534 podStartE2EDuration="2.142157515s" totalImagesPullingTime="478.012981ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:15:38.13982012 +0000 UTC m=+1876.530771596" watchObservedRunningTime="2026-07-24 19:15:38.142157515 +0000 UTC m=+1876.533109036"
Jul 24 19:15:38.156928 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:38.156882    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9" podStartSLOduration=2.156864182 podStartE2EDuration="2.156864182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:15:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:15:38.156251703 +0000 UTC m=+1876.547203180" watchObservedRunningTime="2026-07-24 19:15:38.156864182 +0000 UTC m=+1876.547815661"
Jul 24 19:15:43.049648 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:43.049553    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:15:44.127687 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:44.127648    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:15:49.117158 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:49.117117    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"
Jul 24 19:15:49.117836 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:49.117802    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"
Jul 24 19:15:50.581230 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:50.580989    2563 patch_prober.go:28] interesting pod/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": read tcp 10.134.0.2:33238->10.134.0.44:15021: read: connection reset by peer" start-of-body=
Jul 24 19:15:50.581230 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:50.581057    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr" podUID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": read tcp 10.134.0.2:33238->10.134.0.44:15021: read: connection reset by peer"
Jul 24 19:15:51.180824 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:51.180790    2563 generic.go:356] "Generic (PLEG): container finished" podID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerID="81eba83cc1e39ba94024a04a0f4f1ecfc2a7912ed97d3d5c99225f465770d092" exitCode=0
Jul 24 19:15:51.181002 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:51.180865    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr" event={"ID":"a513b468-4856-4c3f-b22b-061ffd77bd5e","Type":"ContainerDied","Data":"81eba83cc1e39ba94024a04a0f4f1ecfc2a7912ed97d3d5c99225f465770d092"}
Jul 24 19:15:51.181246 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:51.181232    2563 scope.go:117] "RemoveContainer" containerID="81eba83cc1e39ba94024a04a0f4f1ecfc2a7912ed97d3d5c99225f465770d092"
Jul 24 19:15:51.182212 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:51.182185    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:15:51.182316 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:51.182272    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:15:51.182385 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:51.182338    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:15:52.188055 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:52.188013    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr" event={"ID":"a513b468-4856-4c3f-b22b-061ffd77bd5e","Type":"ContainerStarted","Data":"b0d6aecea37fe097049673ae20bd1b5c18432e7ac2bf8124621a016db743e3b9"}
Jul 24 19:15:53.155497 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:53.155456    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:53.155719 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:53.155696    2563 patch_prober.go:28] interesting pod/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 24 19:15:53.155771 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:53.155748    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr" podUID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 24 19:15:54.155815 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:54.155775    2563 patch_prober.go:28] interesting pod/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 24 19:15:54.156416 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:54.155846    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr" podUID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 24 19:15:54.555062 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:54.555038    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:15:55.155702 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:55.155660    2563 patch_prober.go:28] interesting pod/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 24 19:15:55.155885 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:55.155734    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr" podUID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 24 19:15:56.155202 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:56.155168    2563 patch_prober.go:28] interesting pod/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 24 19:15:56.155434 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:56.155244    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr" podUID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 24 19:15:57.159867 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:57.159839    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:57.160257 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:57.160201    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:15:57.160862 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:15:57.160844    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:16:10.363375 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:10.363338    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:16:28.746436 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:28.746386    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:16:30.395963 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.395926    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"]
Jul 24 19:16:30.396541 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.396180    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts" podUID="b989043f-ca9c-4071-a75d-e648724c3b46" containerName="maas-api" containerID="cri-o://fd2f59b625da49195ab5464adfce5f58c0e68a38a127c37dc084022cf5d26cf0" gracePeriod=30
Jul 24 19:16:30.518608 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.518574    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"]
Jul 24 19:16:30.518884 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.518826    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9" podUID="9f68222d-e1d8-45ae-977f-033649f1938b" containerName="payload-processing" containerID="cri-o://a4f683fa70df729e55cc75b2755392a4f2d87a8fece7de9afb53cfb21f21ff37" gracePeriod=30
Jul 24 19:16:30.524245 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.524215    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"]
Jul 24 19:16:30.524667 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.524639    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7" podUID="b6704c83-513e-477e-99e6-5973ace3205a" containerName="payload-pre-processing" containerID="cri-o://fe917c11ff3b9c947a01df7d1542f5fb15ec2b47bdd514fbc122f2b167d73a20" gracePeriod=30
Jul 24 19:16:30.669200 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.669172    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:16:30.798214 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.798189    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"
Jul 24 19:16:30.807698 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.807672    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"
Jul 24 19:16:30.811495 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.811468    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b989043f-ca9c-4071-a75d-e648724c3b46-maas-api-tls\") pod \"b989043f-ca9c-4071-a75d-e648724c3b46\" (UID: \"b989043f-ca9c-4071-a75d-e648724c3b46\") "
Jul 24 19:16:30.811600 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.811541    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b989043f-ca9c-4071-a75d-e648724c3b46-openshift-service-ca\") pod \"b989043f-ca9c-4071-a75d-e648724c3b46\" (UID: \"b989043f-ca9c-4071-a75d-e648724c3b46\") "
Jul 24 19:16:30.811600 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.811593    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wj6c\" (UniqueName: \"kubernetes.io/projected/b989043f-ca9c-4071-a75d-e648724c3b46-kube-api-access-6wj6c\") pod \"b989043f-ca9c-4071-a75d-e648724c3b46\" (UID: \"b989043f-ca9c-4071-a75d-e648724c3b46\") "
Jul 24 19:16:30.811881 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.811857    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b989043f-ca9c-4071-a75d-e648724c3b46-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b989043f-ca9c-4071-a75d-e648724c3b46" (UID: "b989043f-ca9c-4071-a75d-e648724c3b46"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:16:30.813614 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.813571    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b989043f-ca9c-4071-a75d-e648724c3b46-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b989043f-ca9c-4071-a75d-e648724c3b46" (UID: "b989043f-ca9c-4071-a75d-e648724c3b46"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:16:30.813705 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.813668    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b989043f-ca9c-4071-a75d-e648724c3b46-kube-api-access-6wj6c" (OuterVolumeSpecName: "kube-api-access-6wj6c") pod "b989043f-ca9c-4071-a75d-e648724c3b46" (UID: "b989043f-ca9c-4071-a75d-e648724c3b46"). InnerVolumeSpecName "kube-api-access-6wj6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:16:30.913221 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.913105    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f68222d-e1d8-45ae-977f-033649f1938b-plugins-config-volume\") pod \"9f68222d-e1d8-45ae-977f-033649f1938b\" (UID: \"9f68222d-e1d8-45ae-977f-033649f1938b\") "
Jul 24 19:16:30.913221 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.913162    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6704c83-513e-477e-99e6-5973ace3205a-plugins-config-volume\") pod \"b6704c83-513e-477e-99e6-5973ace3205a\" (UID: \"b6704c83-513e-477e-99e6-5973ace3205a\") "
Jul 24 19:16:30.913221 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.913201    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghqsg\" (UniqueName: \"kubernetes.io/projected/9f68222d-e1d8-45ae-977f-033649f1938b-kube-api-access-ghqsg\") pod \"9f68222d-e1d8-45ae-977f-033649f1938b\" (UID: \"9f68222d-e1d8-45ae-977f-033649f1938b\") "
Jul 24 19:16:30.913566 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.913244    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfm8s\" (UniqueName: \"kubernetes.io/projected/b6704c83-513e-477e-99e6-5973ace3205a-kube-api-access-sfm8s\") pod \"b6704c83-513e-477e-99e6-5973ace3205a\" (UID: \"b6704c83-513e-477e-99e6-5973ace3205a\") "
Jul 24 19:16:30.913566 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.913557    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b989043f-ca9c-4071-a75d-e648724c3b46-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:16:30.913674 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.913577    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wj6c\" (UniqueName: \"kubernetes.io/projected/b989043f-ca9c-4071-a75d-e648724c3b46-kube-api-access-6wj6c\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:16:30.913674 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.913592    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b989043f-ca9c-4071-a75d-e648724c3b46-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:16:30.913674 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.913606    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f68222d-e1d8-45ae-977f-033649f1938b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9f68222d-e1d8-45ae-977f-033649f1938b" (UID: "9f68222d-e1d8-45ae-977f-033649f1938b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:16:30.913788 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.913688    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6704c83-513e-477e-99e6-5973ace3205a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b6704c83-513e-477e-99e6-5973ace3205a" (UID: "b6704c83-513e-477e-99e6-5973ace3205a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:16:30.915588 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.915562    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6704c83-513e-477e-99e6-5973ace3205a-kube-api-access-sfm8s" (OuterVolumeSpecName: "kube-api-access-sfm8s") pod "b6704c83-513e-477e-99e6-5973ace3205a" (UID: "b6704c83-513e-477e-99e6-5973ace3205a"). InnerVolumeSpecName "kube-api-access-sfm8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:16:30.915588 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:30.915577    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f68222d-e1d8-45ae-977f-033649f1938b-kube-api-access-ghqsg" (OuterVolumeSpecName: "kube-api-access-ghqsg") pod "9f68222d-e1d8-45ae-977f-033649f1938b" (UID: "9f68222d-e1d8-45ae-977f-033649f1938b"). InnerVolumeSpecName "kube-api-access-ghqsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:16:31.014686 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.014651    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfm8s\" (UniqueName: \"kubernetes.io/projected/b6704c83-513e-477e-99e6-5973ace3205a-kube-api-access-sfm8s\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:16:31.014686 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.014680    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f68222d-e1d8-45ae-977f-033649f1938b-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:16:31.014954 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.014701    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6704c83-513e-477e-99e6-5973ace3205a-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:16:31.014954 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.014712    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghqsg\" (UniqueName: \"kubernetes.io/projected/9f68222d-e1d8-45ae-977f-033649f1938b-kube-api-access-ghqsg\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:16:31.379613 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.379578    2563 generic.go:356] "Generic (PLEG): container finished" podID="b989043f-ca9c-4071-a75d-e648724c3b46" containerID="fd2f59b625da49195ab5464adfce5f58c0e68a38a127c37dc084022cf5d26cf0" exitCode=0
Jul 24 19:16:31.379824 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.379652    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"
Jul 24 19:16:31.379824 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.379675    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts" event={"ID":"b989043f-ca9c-4071-a75d-e648724c3b46","Type":"ContainerDied","Data":"fd2f59b625da49195ab5464adfce5f58c0e68a38a127c37dc084022cf5d26cf0"}
Jul 24 19:16:31.379824 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.379773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts" event={"ID":"b989043f-ca9c-4071-a75d-e648724c3b46","Type":"ContainerDied","Data":"d28461f1947e7ebde6b3dfee8344bb03680a18c3d0a46651b7595061ccaea0f7"}
Jul 24 19:16:31.379824 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.379794    2563 scope.go:117] "RemoveContainer" containerID="fd2f59b625da49195ab5464adfce5f58c0e68a38a127c37dc084022cf5d26cf0"
Jul 24 19:16:31.381122 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.381091    2563 generic.go:356] "Generic (PLEG): container finished" podID="b6704c83-513e-477e-99e6-5973ace3205a" containerID="fe917c11ff3b9c947a01df7d1542f5fb15ec2b47bdd514fbc122f2b167d73a20" exitCode=0
Jul 24 19:16:31.381234 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.381165    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"
Jul 24 19:16:31.381301 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.381165    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7" event={"ID":"b6704c83-513e-477e-99e6-5973ace3205a","Type":"ContainerDied","Data":"fe917c11ff3b9c947a01df7d1542f5fb15ec2b47bdd514fbc122f2b167d73a20"}
Jul 24 19:16:31.381361 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.381296    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7" event={"ID":"b6704c83-513e-477e-99e6-5973ace3205a","Type":"ContainerDied","Data":"859cd7d76793e75e25135b24feb44233fe8ac0ee174bda116db56dc0d5cd38dd"}
Jul 24 19:16:31.382595 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.382574    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f68222d-e1d8-45ae-977f-033649f1938b" containerID="a4f683fa70df729e55cc75b2755392a4f2d87a8fece7de9afb53cfb21f21ff37" exitCode=0
Jul 24 19:16:31.382713 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.382615    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"
Jul 24 19:16:31.382713 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.382646    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9" event={"ID":"9f68222d-e1d8-45ae-977f-033649f1938b","Type":"ContainerDied","Data":"a4f683fa70df729e55cc75b2755392a4f2d87a8fece7de9afb53cfb21f21ff37"}
Jul 24 19:16:31.382713 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.382694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9" event={"ID":"9f68222d-e1d8-45ae-977f-033649f1938b","Type":"ContainerDied","Data":"eb5f22e21d251af67a8b44c1d22a4bf5207b4740859969bc7661bcff30929ae7"}
Jul 24 19:16:31.390634 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.390615    2563 scope.go:117] "RemoveContainer" containerID="fd2f59b625da49195ab5464adfce5f58c0e68a38a127c37dc084022cf5d26cf0"
Jul 24 19:16:31.390920 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:16:31.390898    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd2f59b625da49195ab5464adfce5f58c0e68a38a127c37dc084022cf5d26cf0\": container with ID starting with fd2f59b625da49195ab5464adfce5f58c0e68a38a127c37dc084022cf5d26cf0 not found: ID does not exist" containerID="fd2f59b625da49195ab5464adfce5f58c0e68a38a127c37dc084022cf5d26cf0"
Jul 24 19:16:31.391013 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.390930    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd2f59b625da49195ab5464adfce5f58c0e68a38a127c37dc084022cf5d26cf0"} err="failed to get container status \"fd2f59b625da49195ab5464adfce5f58c0e68a38a127c37dc084022cf5d26cf0\": rpc error: code = NotFound desc = could not find container \"fd2f59b625da49195ab5464adfce5f58c0e68a38a127c37dc084022cf5d26cf0\": container with ID starting with fd2f59b625da49195ab5464adfce5f58c0e68a38a127c37dc084022cf5d26cf0 not found: ID does not exist"
Jul 24 19:16:31.391013 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.390954    2563 scope.go:117] "RemoveContainer" containerID="fe917c11ff3b9c947a01df7d1542f5fb15ec2b47bdd514fbc122f2b167d73a20"
Jul 24 19:16:31.399712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.399429    2563 scope.go:117] "RemoveContainer" containerID="fe917c11ff3b9c947a01df7d1542f5fb15ec2b47bdd514fbc122f2b167d73a20"
Jul 24 19:16:31.399712 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:16:31.399675    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe917c11ff3b9c947a01df7d1542f5fb15ec2b47bdd514fbc122f2b167d73a20\": container with ID starting with fe917c11ff3b9c947a01df7d1542f5fb15ec2b47bdd514fbc122f2b167d73a20 not found: ID does not exist" containerID="fe917c11ff3b9c947a01df7d1542f5fb15ec2b47bdd514fbc122f2b167d73a20"
Jul 24 19:16:31.399712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.399696    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe917c11ff3b9c947a01df7d1542f5fb15ec2b47bdd514fbc122f2b167d73a20"} err="failed to get container status \"fe917c11ff3b9c947a01df7d1542f5fb15ec2b47bdd514fbc122f2b167d73a20\": rpc error: code = NotFound desc = could not find container \"fe917c11ff3b9c947a01df7d1542f5fb15ec2b47bdd514fbc122f2b167d73a20\": container with ID starting with fe917c11ff3b9c947a01df7d1542f5fb15ec2b47bdd514fbc122f2b167d73a20 not found: ID does not exist"
Jul 24 19:16:31.399712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.399711    2563 scope.go:117] "RemoveContainer" containerID="a4f683fa70df729e55cc75b2755392a4f2d87a8fece7de9afb53cfb21f21ff37"
Jul 24 19:16:31.408307 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.408290    2563 scope.go:117] "RemoveContainer" containerID="a4f683fa70df729e55cc75b2755392a4f2d87a8fece7de9afb53cfb21f21ff37"
Jul 24 19:16:31.408610 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:16:31.408578    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4f683fa70df729e55cc75b2755392a4f2d87a8fece7de9afb53cfb21f21ff37\": container with ID starting with a4f683fa70df729e55cc75b2755392a4f2d87a8fece7de9afb53cfb21f21ff37 not found: ID does not exist" containerID="a4f683fa70df729e55cc75b2755392a4f2d87a8fece7de9afb53cfb21f21ff37"
Jul 24 19:16:31.408712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.408611    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f683fa70df729e55cc75b2755392a4f2d87a8fece7de9afb53cfb21f21ff37"} err="failed to get container status \"a4f683fa70df729e55cc75b2755392a4f2d87a8fece7de9afb53cfb21f21ff37\": rpc error: code = NotFound desc = could not find container \"a4f683fa70df729e55cc75b2755392a4f2d87a8fece7de9afb53cfb21f21ff37\": container with ID starting with a4f683fa70df729e55cc75b2755392a4f2d87a8fece7de9afb53cfb21f21ff37 not found: ID does not exist"
Jul 24 19:16:31.408840 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.408822    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"]
Jul 24 19:16:31.416316 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.416288    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7baf010d-5b45cbf8c9-r2nts"]
Jul 24 19:16:31.425136 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.425114    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"]
Jul 24 19:16:31.430681 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.430661    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7baf010d-7765c67658-6cqc9"]
Jul 24 19:16:31.440729 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.440710    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"]
Jul 24 19:16:31.446582 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:31.446562    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7baf010d-7dcbc9bb84-42hx7"]
Jul 24 19:16:32.203797 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:32.203768    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f68222d-e1d8-45ae-977f-033649f1938b" path="/var/lib/kubelet/pods/9f68222d-e1d8-45ae-977f-033649f1938b/volumes"
Jul 24 19:16:32.204113 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:32.204100    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6704c83-513e-477e-99e6-5973ace3205a" path="/var/lib/kubelet/pods/b6704c83-513e-477e-99e6-5973ace3205a/volumes"
Jul 24 19:16:32.204446 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:32.204434    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b989043f-ca9c-4071-a75d-e648724c3b46" path="/var/lib/kubelet/pods/b989043f-ca9c-4071-a75d-e648724c3b46/volumes"
Jul 24 19:16:39.447797 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:16:39.447762    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:17:26.097423 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:26.097366    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"]
Jul 24 19:17:26.097894 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:26.097600    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr" podUID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerName="istio-proxy" containerID="cri-o://b0d6aecea37fe097049673ae20bd1b5c18432e7ac2bf8124621a016db743e3b9" gracePeriod=30
Jul 24 19:17:27.163126 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:27.163090    2563 patch_prober.go:28] interesting pod/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 19:17:27.163534 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:27.163167    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr" podUID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 19:17:30.153623 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:30.153587    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:17:31.353348 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.353327    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:17:31.512699 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.512621    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-workload-certs\") pod \"a513b468-4856-4c3f-b22b-061ffd77bd5e\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") "
Jul 24 19:17:31.512699 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.512653    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-data\") pod \"a513b468-4856-4c3f-b22b-061ffd77bd5e\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") "
Jul 24 19:17:31.512699 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.512679    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-token\") pod \"a513b468-4856-4c3f-b22b-061ffd77bd5e\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") "
Jul 24 19:17:31.512985 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.512712    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-credential-socket\") pod \"a513b468-4856-4c3f-b22b-061ffd77bd5e\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") "
Jul 24 19:17:31.512985 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.512736    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-workload-socket\") pod \"a513b468-4856-4c3f-b22b-061ffd77bd5e\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") "
Jul 24 19:17:31.512985 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.512787    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjcc5\" (UniqueName: \"kubernetes.io/projected/a513b468-4856-4c3f-b22b-061ffd77bd5e-kube-api-access-sjcc5\") pod \"a513b468-4856-4c3f-b22b-061ffd77bd5e\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") "
Jul 24 19:17:31.512985 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.512834    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-podinfo\") pod \"a513b468-4856-4c3f-b22b-061ffd77bd5e\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") "
Jul 24 19:17:31.512985 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.512879    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a513b468-4856-4c3f-b22b-061ffd77bd5e-istiod-ca-cert\") pod \"a513b468-4856-4c3f-b22b-061ffd77bd5e\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") "
Jul 24 19:17:31.512985 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.512903    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-envoy\") pod \"a513b468-4856-4c3f-b22b-061ffd77bd5e\" (UID: \"a513b468-4856-4c3f-b22b-061ffd77bd5e\") "
Jul 24 19:17:31.512985 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.512947    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a513b468-4856-4c3f-b22b-061ffd77bd5e" (UID: "a513b468-4856-4c3f-b22b-061ffd77bd5e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:17:31.513369 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.513057    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a513b468-4856-4c3f-b22b-061ffd77bd5e" (UID: "a513b468-4856-4c3f-b22b-061ffd77bd5e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:17:31.513369 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.513066    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a513b468-4856-4c3f-b22b-061ffd77bd5e" (UID: "a513b468-4856-4c3f-b22b-061ffd77bd5e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:17:31.513369 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.513201    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-credential-socket\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:17:31.513369 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.513249    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-workload-socket\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:17:31.513369 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.513267    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-workload-certs\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:17:31.513369 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.513375    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a513b468-4856-4c3f-b22b-061ffd77bd5e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a513b468-4856-4c3f-b22b-061ffd77bd5e" (UID: "a513b468-4856-4c3f-b22b-061ffd77bd5e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:17:31.513887 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.513704    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-data" (OuterVolumeSpecName: "istio-data") pod "a513b468-4856-4c3f-b22b-061ffd77bd5e" (UID: "a513b468-4856-4c3f-b22b-061ffd77bd5e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:17:31.515074 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.515046    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-token" (OuterVolumeSpecName: "istio-token") pod "a513b468-4856-4c3f-b22b-061ffd77bd5e" (UID: "a513b468-4856-4c3f-b22b-061ffd77bd5e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:17:31.515182 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.515103    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a513b468-4856-4c3f-b22b-061ffd77bd5e" (UID: "a513b468-4856-4c3f-b22b-061ffd77bd5e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 19:17:31.515182 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.515127    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a513b468-4856-4c3f-b22b-061ffd77bd5e-kube-api-access-sjcc5" (OuterVolumeSpecName: "kube-api-access-sjcc5") pod "a513b468-4856-4c3f-b22b-061ffd77bd5e" (UID: "a513b468-4856-4c3f-b22b-061ffd77bd5e"). InnerVolumeSpecName "kube-api-access-sjcc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:17:31.515261 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.515176    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a513b468-4856-4c3f-b22b-061ffd77bd5e" (UID: "a513b468-4856-4c3f-b22b-061ffd77bd5e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:17:31.614059 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.614010    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-podinfo\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:17:31.614059 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.614051    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a513b468-4856-4c3f-b22b-061ffd77bd5e-istiod-ca-cert\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:17:31.614059 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.614060    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-envoy\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:17:31.614059 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.614068    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-data\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:17:31.614059 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.614075    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a513b468-4856-4c3f-b22b-061ffd77bd5e-istio-token\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:17:31.614059 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.614083    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjcc5\" (UniqueName: \"kubernetes.io/projected/a513b468-4856-4c3f-b22b-061ffd77bd5e-kube-api-access-sjcc5\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:17:31.675951 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.675913    2563 generic.go:356] "Generic (PLEG): container finished" podID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerID="b0d6aecea37fe097049673ae20bd1b5c18432e7ac2bf8124621a016db743e3b9" exitCode=0
Jul 24 19:17:31.676109 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.675958    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr" event={"ID":"a513b468-4856-4c3f-b22b-061ffd77bd5e","Type":"ContainerDied","Data":"b0d6aecea37fe097049673ae20bd1b5c18432e7ac2bf8124621a016db743e3b9"}
Jul 24 19:17:31.676109 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.675971    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"
Jul 24 19:17:31.676109 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.676013    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr" event={"ID":"a513b468-4856-4c3f-b22b-061ffd77bd5e","Type":"ContainerDied","Data":"c7e65fdfdbf2c05cf848f02705b236d19a9e278cc2d9bcbad70a4b786604ec91"}
Jul 24 19:17:31.676109 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.676045    2563 scope.go:117] "RemoveContainer" containerID="b0d6aecea37fe097049673ae20bd1b5c18432e7ac2bf8124621a016db743e3b9"
Jul 24 19:17:31.685529 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.685493    2563 scope.go:117] "RemoveContainer" containerID="81eba83cc1e39ba94024a04a0f4f1ecfc2a7912ed97d3d5c99225f465770d092"
Jul 24 19:17:31.693530 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.693512    2563 scope.go:117] "RemoveContainer" containerID="b0d6aecea37fe097049673ae20bd1b5c18432e7ac2bf8124621a016db743e3b9"
Jul 24 19:17:31.693826 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:17:31.693806    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0d6aecea37fe097049673ae20bd1b5c18432e7ac2bf8124621a016db743e3b9\": container with ID starting with b0d6aecea37fe097049673ae20bd1b5c18432e7ac2bf8124621a016db743e3b9 not found: ID does not exist" containerID="b0d6aecea37fe097049673ae20bd1b5c18432e7ac2bf8124621a016db743e3b9"
Jul 24 19:17:31.693872 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.693836    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0d6aecea37fe097049673ae20bd1b5c18432e7ac2bf8124621a016db743e3b9"} err="failed to get container status \"b0d6aecea37fe097049673ae20bd1b5c18432e7ac2bf8124621a016db743e3b9\": rpc error: code = NotFound desc = could not find container \"b0d6aecea37fe097049673ae20bd1b5c18432e7ac2bf8124621a016db743e3b9\": container with ID starting with b0d6aecea37fe097049673ae20bd1b5c18432e7ac2bf8124621a016db743e3b9 not found: ID does not exist"
Jul 24 19:17:31.693872 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.693856    2563 scope.go:117] "RemoveContainer" containerID="81eba83cc1e39ba94024a04a0f4f1ecfc2a7912ed97d3d5c99225f465770d092"
Jul 24 19:17:31.694096 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:17:31.694078    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81eba83cc1e39ba94024a04a0f4f1ecfc2a7912ed97d3d5c99225f465770d092\": container with ID starting with 81eba83cc1e39ba94024a04a0f4f1ecfc2a7912ed97d3d5c99225f465770d092 not found: ID does not exist" containerID="81eba83cc1e39ba94024a04a0f4f1ecfc2a7912ed97d3d5c99225f465770d092"
Jul 24 19:17:31.694149 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.694104    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81eba83cc1e39ba94024a04a0f4f1ecfc2a7912ed97d3d5c99225f465770d092"} err="failed to get container status \"81eba83cc1e39ba94024a04a0f4f1ecfc2a7912ed97d3d5c99225f465770d092\": rpc error: code = NotFound desc = could not find container \"81eba83cc1e39ba94024a04a0f4f1ecfc2a7912ed97d3d5c99225f465770d092\": container with ID starting with 81eba83cc1e39ba94024a04a0f4f1ecfc2a7912ed97d3d5c99225f465770d092 not found: ID does not exist"
Jul 24 19:17:31.697840 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.697817    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"]
Jul 24 19:17:31.702206 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:31.702183    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7baf010d-openshift-default-7cc94888f6-xhqdr"]
Jul 24 19:17:32.204797 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:32.204764    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a513b468-4856-4c3f-b22b-061ffd77bd5e" path="/var/lib/kubelet/pods/a513b468-4856-4c3f-b22b-061ffd77bd5e/volumes"
Jul 24 19:17:43.322119 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:17:43.322080    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:18:21.129344 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.129312    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"]
Jul 24 19:18:21.129837 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.129615    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6704c83-513e-477e-99e6-5973ace3205a" containerName="payload-pre-processing"
Jul 24 19:18:21.129837 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.129627    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6704c83-513e-477e-99e6-5973ace3205a" containerName="payload-pre-processing"
Jul 24 19:18:21.129837 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.129635    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b989043f-ca9c-4071-a75d-e648724c3b46" containerName="maas-api"
Jul 24 19:18:21.129837 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.129641    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b989043f-ca9c-4071-a75d-e648724c3b46" containerName="maas-api"
Jul 24 19:18:21.129837 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.129652    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f68222d-e1d8-45ae-977f-033649f1938b" containerName="payload-processing"
Jul 24 19:18:21.129837 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.129658    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f68222d-e1d8-45ae-977f-033649f1938b" containerName="payload-processing"
Jul 24 19:18:21.129837 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.129667    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerName="istio-proxy"
Jul 24 19:18:21.129837 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.129673    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerName="istio-proxy"
Jul 24 19:18:21.129837 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.129719    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerName="istio-proxy"
Jul 24 19:18:21.129837 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.129725    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6704c83-513e-477e-99e6-5973ace3205a" containerName="payload-pre-processing"
Jul 24 19:18:21.129837 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.129732    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f68222d-e1d8-45ae-977f-033649f1938b" containerName="payload-processing"
Jul 24 19:18:21.129837 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.129740    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b989043f-ca9c-4071-a75d-e648724c3b46" containerName="maas-api"
Jul 24 19:18:21.129837 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.129746    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerName="istio-proxy"
Jul 24 19:18:21.132637 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.132619    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.135264 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.135244    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-950cf0-openshift-default-dockercfg-8wkbm\""
Jul 24 19:18:21.144147 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.144121    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"]
Jul 24 19:18:21.240754 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.240716    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-workload-certs\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.240960 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.240774    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7594d561-a553-4388-b2dd-5b4709e05008-istio-token\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.240960 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.240793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7594d561-a553-4388-b2dd-5b4709e05008-istio-podinfo\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.240960 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.240811    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-istio-envoy\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.240960 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.240911    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7594d561-a553-4388-b2dd-5b4709e05008-istiod-ca-cert\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.241169 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.240998    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-credential-socket\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.241169 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.241032    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-istio-data\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.241169 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.241065    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-workload-socket\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.241169 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.241092    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhp2j\" (UniqueName: \"kubernetes.io/projected/7594d561-a553-4388-b2dd-5b4709e05008-kube-api-access-xhp2j\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.342140 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.342104    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-workload-socket\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.342140 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.342139    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhp2j\" (UniqueName: \"kubernetes.io/projected/7594d561-a553-4388-b2dd-5b4709e05008-kube-api-access-xhp2j\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.342373 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.342167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-workload-certs\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.342373 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.342216    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7594d561-a553-4388-b2dd-5b4709e05008-istio-token\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.342373 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.342235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7594d561-a553-4388-b2dd-5b4709e05008-istio-podinfo\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.342373 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.342255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-istio-envoy\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.342625 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.342439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7594d561-a553-4388-b2dd-5b4709e05008-istiod-ca-cert\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.342625 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.342517    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-credential-socket\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.342625 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.342546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-istio-data\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.342771 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.342641    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-workload-socket\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.342771 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.342716    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-workload-certs\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.342946 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.342924    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-istio-data\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.343023 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.342940    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-credential-socket\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.343221 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.343201    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7594d561-a553-4388-b2dd-5b4709e05008-istiod-ca-cert\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.344822 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.344801    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-istio-envoy\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.344953 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.344936    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7594d561-a553-4388-b2dd-5b4709e05008-istio-podinfo\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.349611 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.349591    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7594d561-a553-4388-b2dd-5b4709e05008-istio-token\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.349913 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.349866    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhp2j\" (UniqueName: \"kubernetes.io/projected/7594d561-a553-4388-b2dd-5b4709e05008-kube-api-access-xhp2j\") pod \"e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") " pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.443063 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.442971    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:21.583563 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.583538    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"]
Jul 24 19:18:21.585421 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:18:21.585363    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7594d561_a553_4388_b2dd_5b4709e05008.slice/crio-074dd521d13c86359deefb0b9df460f1e168682310e88d36fb683c9b4a70ffb7 WatchSource:0}: Error finding container 074dd521d13c86359deefb0b9df460f1e168682310e88d36fb683c9b4a70ffb7: Status 404 returned error can't find the container with id 074dd521d13c86359deefb0b9df460f1e168682310e88d36fb683c9b4a70ffb7
Jul 24 19:18:21.587651 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.587610    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:18:21.587732 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.587699    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:18:21.587774 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.587741    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:18:21.918345 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.918309    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl" event={"ID":"7594d561-a553-4388-b2dd-5b4709e05008","Type":"ContainerStarted","Data":"499bd3e14dfd024ddd700da74bc0980468ac1c354966462a54413a8c32d010ee"}
Jul 24 19:18:21.918534 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.918367    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl" event={"ID":"7594d561-a553-4388-b2dd-5b4709e05008","Type":"ContainerStarted","Data":"074dd521d13c86359deefb0b9df460f1e168682310e88d36fb683c9b4a70ffb7"}
Jul 24 19:18:21.938142 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:21.938080    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl" podStartSLOduration=0.938065384 podStartE2EDuration="938.065384ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:18:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:18:21.935983097 +0000 UTC m=+2040.326934589" watchObservedRunningTime="2026-07-24 19:18:21.938065384 +0000 UTC m=+2040.329016861"
Jul 24 19:18:22.443573 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:22.443532    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:22.448263 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:22.448236    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:22.923543 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:22.923511    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:22.924536 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:22.924511    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:23.950495 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:23.950460    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"]
Jul 24 19:18:23.951019 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:23.950765    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerName="istio-proxy"
Jul 24 19:18:23.951019 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:23.950777    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a513b468-4856-4c3f-b22b-061ffd77bd5e" containerName="istio-proxy"
Jul 24 19:18:23.953682 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:23.953660    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:18:23.956358 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:23.956336    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 19:18:23.957057 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:23.957037    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-950cf0-serving-cert\""
Jul 24 19:18:23.957057 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:23.957050    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 19:18:23.957217 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:23.957071    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cvjqn\""
Jul 24 19:18:23.965141 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:23.965114    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"]
Jul 24 19:18:24.064555 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.064512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5rjt\" (UniqueName: \"kubernetes.io/projected/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-kube-api-access-f5rjt\") pod \"maas-api-e2e-inf-a-950cf0-569d995975-4xffp\" (UID: \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:18:24.064740 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.064584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-950cf0-569d995975-4xffp\" (UID: \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:18:24.064740 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.064692    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-maas-api-tls\") pod \"maas-api-e2e-inf-a-950cf0-569d995975-4xffp\" (UID: \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:18:24.165857 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.165822    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5rjt\" (UniqueName: \"kubernetes.io/projected/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-kube-api-access-f5rjt\") pod \"maas-api-e2e-inf-a-950cf0-569d995975-4xffp\" (UID: \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:18:24.166045 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.165870    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-950cf0-569d995975-4xffp\" (UID: \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:18:24.166045 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.165906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-maas-api-tls\") pod \"maas-api-e2e-inf-a-950cf0-569d995975-4xffp\" (UID: \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:18:24.166560 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.166537    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-950cf0-569d995975-4xffp\" (UID: \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:18:24.168512 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.168496    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-maas-api-tls\") pod \"maas-api-e2e-inf-a-950cf0-569d995975-4xffp\" (UID: \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:18:24.175601 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.175577    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5rjt\" (UniqueName: \"kubernetes.io/projected/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-kube-api-access-f5rjt\") pod \"maas-api-e2e-inf-a-950cf0-569d995975-4xffp\" (UID: \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:18:24.266210 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.266114    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:18:24.401271 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.401246    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"]
Jul 24 19:18:24.403041 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:18:24.403017    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72d2bb9c_1735_49fc_9ca1_27645fa5cea1.slice/crio-e47aa231ca3ce81fcd42bc41373adfc871709ce92e6e7a99c3d32b430761bea0 WatchSource:0}: Error finding container e47aa231ca3ce81fcd42bc41373adfc871709ce92e6e7a99c3d32b430761bea0: Status 404 returned error can't find the container with id e47aa231ca3ce81fcd42bc41373adfc871709ce92e6e7a99c3d32b430761bea0
Jul 24 19:18:24.765459 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.765423    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"]
Jul 24 19:18:24.769871 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.769849    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"
Jul 24 19:18:24.772366 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.771975    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-950cf0\""
Jul 24 19:18:24.772366 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.772317    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-950cf0-dockercfg-gzlrl\""
Jul 24 19:18:24.780914 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.780884    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"]
Jul 24 19:18:24.783366 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.783343    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"]
Jul 24 19:18:24.786686 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.786666    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"
Jul 24 19:18:24.796486 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.796458    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"]
Jul 24 19:18:24.871629 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.871597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dd99a6f-a52b-439c-8c2c-1fc298a13b42-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw\" (UID: \"0dd99a6f-a52b-439c-8c2c-1fc298a13b42\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"
Jul 24 19:18:24.871786 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.871642    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j92pj\" (UniqueName: \"kubernetes.io/projected/0dd99a6f-a52b-439c-8c2c-1fc298a13b42-kube-api-access-j92pj\") pod \"payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw\" (UID: \"0dd99a6f-a52b-439c-8c2c-1fc298a13b42\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"
Jul 24 19:18:24.934661 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.934630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp" event={"ID":"72d2bb9c-1735-49fc-9ca1-27645fa5cea1","Type":"ContainerStarted","Data":"37af2312ab5158857efb0b9257df32c88ca17faf7833281f7ec68f1603272dc3"}
Jul 24 19:18:24.934829 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.934682    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp" event={"ID":"72d2bb9c-1735-49fc-9ca1-27645fa5cea1","Type":"ContainerStarted","Data":"e47aa231ca3ce81fcd42bc41373adfc871709ce92e6e7a99c3d32b430761bea0"}
Jul 24 19:18:24.934894 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.934871    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:18:24.953771 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.953711    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp" podStartSLOduration=1.514669746 podStartE2EDuration="1.95369665s" totalImagesPullingTime="439.026904ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:18:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:18:24.952127996 +0000 UTC m=+2043.343079474" watchObservedRunningTime="2026-07-24 19:18:24.95369665 +0000 UTC m=+2043.344648127"
Jul 24 19:18:24.972585 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.972558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwrcr\" (UniqueName: \"kubernetes.io/projected/9d14eed6-d34f-49c2-bdfb-0932e02c526f-kube-api-access-nwrcr\") pod \"payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk\" (UID: \"9d14eed6-d34f-49c2-bdfb-0932e02c526f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"
Jul 24 19:18:24.972698 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.972597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dd99a6f-a52b-439c-8c2c-1fc298a13b42-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw\" (UID: \"0dd99a6f-a52b-439c-8c2c-1fc298a13b42\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"
Jul 24 19:18:24.972698 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.972615    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j92pj\" (UniqueName: \"kubernetes.io/projected/0dd99a6f-a52b-439c-8c2c-1fc298a13b42-kube-api-access-j92pj\") pod \"payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw\" (UID: \"0dd99a6f-a52b-439c-8c2c-1fc298a13b42\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"
Jul 24 19:18:24.972795 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.972720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d14eed6-d34f-49c2-bdfb-0932e02c526f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk\" (UID: \"9d14eed6-d34f-49c2-bdfb-0932e02c526f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"
Jul 24 19:18:24.973291 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.973273    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dd99a6f-a52b-439c-8c2c-1fc298a13b42-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw\" (UID: \"0dd99a6f-a52b-439c-8c2c-1fc298a13b42\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"
Jul 24 19:18:24.981350 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:24.981327    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j92pj\" (UniqueName: \"kubernetes.io/projected/0dd99a6f-a52b-439c-8c2c-1fc298a13b42-kube-api-access-j92pj\") pod \"payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw\" (UID: \"0dd99a6f-a52b-439c-8c2c-1fc298a13b42\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"
Jul 24 19:18:25.073784 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.073746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d14eed6-d34f-49c2-bdfb-0932e02c526f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk\" (UID: \"9d14eed6-d34f-49c2-bdfb-0932e02c526f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"
Jul 24 19:18:25.073972 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.073924    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwrcr\" (UniqueName: \"kubernetes.io/projected/9d14eed6-d34f-49c2-bdfb-0932e02c526f-kube-api-access-nwrcr\") pod \"payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk\" (UID: \"9d14eed6-d34f-49c2-bdfb-0932e02c526f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"
Jul 24 19:18:25.074584 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.074564    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d14eed6-d34f-49c2-bdfb-0932e02c526f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk\" (UID: \"9d14eed6-d34f-49c2-bdfb-0932e02c526f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"
Jul 24 19:18:25.082360 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.082335    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwrcr\" (UniqueName: \"kubernetes.io/projected/9d14eed6-d34f-49c2-bdfb-0932e02c526f-kube-api-access-nwrcr\") pod \"payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk\" (UID: \"9d14eed6-d34f-49c2-bdfb-0932e02c526f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"
Jul 24 19:18:25.086141 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.086123    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"
Jul 24 19:18:25.101039 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.101020    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"
Jul 24 19:18:25.233940 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.233914    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"]
Jul 24 19:18:25.235977 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:18:25.235944    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dd99a6f_a52b_439c_8c2c_1fc298a13b42.slice/crio-2038fdcb077ab3f6bf94111db49a441ae5d3ae3f2fbba18221c0e380d512915a WatchSource:0}: Error finding container 2038fdcb077ab3f6bf94111db49a441ae5d3ae3f2fbba18221c0e380d512915a: Status 404 returned error can't find the container with id 2038fdcb077ab3f6bf94111db49a441ae5d3ae3f2fbba18221c0e380d512915a
Jul 24 19:18:25.258537 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.258450    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"]
Jul 24 19:18:25.259947 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:18:25.259923    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d14eed6_d34f_49c2_bdfb_0932e02c526f.slice/crio-624e1becc41525dad3e51372fde9e96b8d8ccf32435d4dc344a89f745b35cea3 WatchSource:0}: Error finding container 624e1becc41525dad3e51372fde9e96b8d8ccf32435d4dc344a89f745b35cea3: Status 404 returned error can't find the container with id 624e1becc41525dad3e51372fde9e96b8d8ccf32435d4dc344a89f745b35cea3
Jul 24 19:18:25.941660 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.941619    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw" event={"ID":"0dd99a6f-a52b-439c-8c2c-1fc298a13b42","Type":"ContainerStarted","Data":"eb5095f07e64874eb683b61bf50294750392826f6b5698f0c68eb0e0f0d21dce"}
Jul 24 19:18:25.941894 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.941689    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"
Jul 24 19:18:25.941894 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.941714    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw" event={"ID":"0dd99a6f-a52b-439c-8c2c-1fc298a13b42","Type":"ContainerStarted","Data":"2038fdcb077ab3f6bf94111db49a441ae5d3ae3f2fbba18221c0e380d512915a"}
Jul 24 19:18:25.943242 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.943217    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk" event={"ID":"9d14eed6-d34f-49c2-bdfb-0932e02c526f","Type":"ContainerStarted","Data":"d8b4dd3b3e805a528d46a49bd48faeadb1a4b2ec2a0e87e98b45ef4a48e4ccb7"}
Jul 24 19:18:25.943376 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.943257    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk" event={"ID":"9d14eed6-d34f-49c2-bdfb-0932e02c526f","Type":"ContainerStarted","Data":"624e1becc41525dad3e51372fde9e96b8d8ccf32435d4dc344a89f745b35cea3"}
Jul 24 19:18:25.943376 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.943372    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"
Jul 24 19:18:25.959009 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.958968    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw" podStartSLOduration=1.958952464 podStartE2EDuration="1.958952464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:18:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:18:25.957692253 +0000 UTC m=+2044.348643730" watchObservedRunningTime="2026-07-24 19:18:25.958952464 +0000 UTC m=+2044.349903941"
Jul 24 19:18:25.974977 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:25.974927    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk" podStartSLOduration=1.974912886 podStartE2EDuration="1.974912886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:18:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:18:25.973641509 +0000 UTC m=+2044.364593011" watchObservedRunningTime="2026-07-24 19:18:25.974912886 +0000 UTC m=+2044.365864362"
Jul 24 19:18:29.750347 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:29.750304    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:18:30.949145 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:30.949112    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:18:32.174308 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:32.174277    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"]
Jul 24 19:18:32.178742 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:32.178723    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:18:32.181967 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:32.181945    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-948d66-serving-cert\""
Jul 24 19:18:32.192439 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:32.192414    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"]
Jul 24 19:18:32.243067 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:32.243036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23f99621-316f-46cd-98fd-7533dc9e2ca2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq\" (UID: \"23f99621-316f-46cd-98fd-7533dc9e2ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:18:32.243067 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:32.243075    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23f99621-316f-46cd-98fd-7533dc9e2ca2-maas-api-tls\") pod \"maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq\" (UID: \"23f99621-316f-46cd-98fd-7533dc9e2ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:18:32.243298 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:32.243152    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgf7j\" (UniqueName: \"kubernetes.io/projected/23f99621-316f-46cd-98fd-7533dc9e2ca2-kube-api-access-jgf7j\") pod \"maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq\" (UID: \"23f99621-316f-46cd-98fd-7533dc9e2ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:18:32.343807 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:32.343771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23f99621-316f-46cd-98fd-7533dc9e2ca2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq\" (UID: \"23f99621-316f-46cd-98fd-7533dc9e2ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:18:32.343807 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:32.343812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23f99621-316f-46cd-98fd-7533dc9e2ca2-maas-api-tls\") pod \"maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq\" (UID: \"23f99621-316f-46cd-98fd-7533dc9e2ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:18:32.344097 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:32.343852    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgf7j\" (UniqueName: \"kubernetes.io/projected/23f99621-316f-46cd-98fd-7533dc9e2ca2-kube-api-access-jgf7j\") pod \"maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq\" (UID: \"23f99621-316f-46cd-98fd-7533dc9e2ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:18:32.344097 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:18:32.343917    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-serving-cert: secret "maas-api-e2e-inf-b-948d66-serving-cert" not found
Jul 24 19:18:32.344097 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:18:32.343984    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23f99621-316f-46cd-98fd-7533dc9e2ca2-maas-api-tls podName:23f99621-316f-46cd-98fd-7533dc9e2ca2 nodeName:}" failed. No retries permitted until 2026-07-24 19:18:32.843967741 +0000 UTC m=+2051.234919196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/23f99621-316f-46cd-98fd-7533dc9e2ca2-maas-api-tls") pod "maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq" (UID: "23f99621-316f-46cd-98fd-7533dc9e2ca2") : secret "maas-api-e2e-inf-b-948d66-serving-cert" not found
Jul 24 19:18:32.344420 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:32.344374    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23f99621-316f-46cd-98fd-7533dc9e2ca2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq\" (UID: \"23f99621-316f-46cd-98fd-7533dc9e2ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:18:32.352767 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:32.352743    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgf7j\" (UniqueName: \"kubernetes.io/projected/23f99621-316f-46cd-98fd-7533dc9e2ca2-kube-api-access-jgf7j\") pod \"maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq\" (UID: \"23f99621-316f-46cd-98fd-7533dc9e2ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:18:32.847652 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:32.847620    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23f99621-316f-46cd-98fd-7533dc9e2ca2-maas-api-tls\") pod \"maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq\" (UID: \"23f99621-316f-46cd-98fd-7533dc9e2ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:18:32.850249 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:32.850221    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23f99621-316f-46cd-98fd-7533dc9e2ca2-maas-api-tls\") pod \"maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq\" (UID: \"23f99621-316f-46cd-98fd-7533dc9e2ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:18:33.089183 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:33.089148    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:18:33.280019 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:33.279995    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"]
Jul 24 19:18:33.281825 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:18:33.281794    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23f99621_316f_46cd_98fd_7533dc9e2ca2.slice/crio-4468e22df1925c7bb6fb961cfb2c74f69d4ec10358689e962023c0a598daa5bd WatchSource:0}: Error finding container 4468e22df1925c7bb6fb961cfb2c74f69d4ec10358689e962023c0a598daa5bd: Status 404 returned error can't find the container with id 4468e22df1925c7bb6fb961cfb2c74f69d4ec10358689e962023c0a598daa5bd
Jul 24 19:18:33.987574 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:33.987472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq" event={"ID":"23f99621-316f-46cd-98fd-7533dc9e2ca2","Type":"ContainerStarted","Data":"925f09dc20427144ca3b11cd932701dc5f48841a8654eb327c48f185ac9e7046"}
Jul 24 19:18:33.987574 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:33.987526    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq" event={"ID":"23f99621-316f-46cd-98fd-7533dc9e2ca2","Type":"ContainerStarted","Data":"4468e22df1925c7bb6fb961cfb2c74f69d4ec10358689e962023c0a598daa5bd"}
Jul 24 19:18:33.987776 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:33.987580    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:18:34.005587 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:34.005533    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq" podStartSLOduration=1.694040406 podStartE2EDuration="2.005516603s" totalImagesPullingTime="311.476197ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:18:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:18:34.003457539 +0000 UTC m=+2052.394409029" watchObservedRunningTime="2026-07-24 19:18:34.005516603 +0000 UTC m=+2052.396468080"
Jul 24 19:18:36.949547 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:36.949506    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw" podUID="0dd99a6f-a52b-439c-8c2c-1fc298a13b42" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.50:9004: connect: connection refused"
Jul 24 19:18:36.950033 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:36.950013    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"
Jul 24 19:18:37.924602 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:37.924562    2563 patch_prober.go:28] interesting pod/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 24 19:18:37.924796 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:37.924624    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl" podUID="7594d561-a553-4388-b2dd-5b4709e05008" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 24 19:18:38.010644 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:38.010560    2563 generic.go:356] "Generic (PLEG): container finished" podID="7594d561-a553-4388-b2dd-5b4709e05008" containerID="499bd3e14dfd024ddd700da74bc0980468ac1c354966462a54413a8c32d010ee" exitCode=0
Jul 24 19:18:38.010644 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:38.010600    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl" event={"ID":"7594d561-a553-4388-b2dd-5b4709e05008","Type":"ContainerDied","Data":"499bd3e14dfd024ddd700da74bc0980468ac1c354966462a54413a8c32d010ee"}
Jul 24 19:18:38.011044 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:38.010954    2563 scope.go:117] "RemoveContainer" containerID="499bd3e14dfd024ddd700da74bc0980468ac1c354966462a54413a8c32d010ee"
Jul 24 19:18:38.011971 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:38.011943    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:18:38.012042 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:38.012007    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:18:38.012042 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:38.012038    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:18:39.020376 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:39.020326    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl" event={"ID":"7594d561-a553-4388-b2dd-5b4709e05008","Type":"ContainerStarted","Data":"2c449d689f78c4fa34bf61357bfbb673cda892ade72d7326fa99ff9ec5ee5062"}
Jul 24 19:18:39.444132 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:39.444095    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:39.444414 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:39.444368    2563 patch_prober.go:28] interesting pod/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 24 19:18:39.444484 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:39.444438    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl" podUID="7594d561-a553-4388-b2dd-5b4709e05008" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 24 19:18:39.999263 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:39.999234    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:18:40.443610 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:40.443567    2563 patch_prober.go:28] interesting pod/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 24 19:18:40.444101 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:40.443643    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl" podUID="7594d561-a553-4388-b2dd-5b4709e05008" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 24 19:18:41.444257 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:41.444220    2563 patch_prober.go:28] interesting pod/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 24 19:18:41.444670 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:41.444279    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl" podUID="7594d561-a553-4388-b2dd-5b4709e05008" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 24 19:18:42.361058 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:42.360972    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:18:42.444303 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:42.444263    2563 patch_prober.go:28] interesting pod/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 24 19:18:42.444824 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:42.444333    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl" podUID="7594d561-a553-4388-b2dd-5b4709e05008" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 24 19:18:43.447449 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:43.447382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:43.447890 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:43.447689    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:43.448443 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:43.448426    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:18:46.949770 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:46.949738    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"
Jul 24 19:18:57.471006 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:18:57.470963    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:19:08.553256 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:19:08.553214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:19:19.558874 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:19:19.558838    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:19:22.228671 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:19:22.228642    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 19:19:22.238515 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:19:22.238487    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 19:19:33.551446 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:19:33.551387    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:20:07.295388 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:07.295349    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:20:12.952571 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:12.952466    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"]
Jul 24 19:20:12.953032 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:12.952710    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp" podUID="72d2bb9c-1735-49fc-9ca1-27645fa5cea1" containerName="maas-api" containerID="cri-o://37af2312ab5158857efb0b9257df32c88ca17faf7833281f7ec68f1603272dc3" gracePeriod=30
Jul 24 19:20:13.014211 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.014172    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"]
Jul 24 19:20:13.014478 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.014449    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk" podUID="9d14eed6-d34f-49c2-bdfb-0932e02c526f" containerName="payload-processing" containerID="cri-o://d8b4dd3b3e805a528d46a49bd48faeadb1a4b2ec2a0e87e98b45ef4a48e4ccb7" gracePeriod=30
Jul 24 19:20:13.024545 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.024516    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"]
Jul 24 19:20:13.024732 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.024710    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw" podUID="0dd99a6f-a52b-439c-8c2c-1fc298a13b42" containerName="payload-pre-processing" containerID="cri-o://eb5095f07e64874eb683b61bf50294750392826f6b5698f0c68eb0e0f0d21dce" gracePeriod=30
Jul 24 19:20:13.298281 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.298251    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"
Jul 24 19:20:13.301090 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.301069    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:20:13.335225 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.333822    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"
Jul 24 19:20:13.340617 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.340590    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-maas-api-tls\") pod \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\" (UID: \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\") "
Jul 24 19:20:13.340765 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.340631    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5rjt\" (UniqueName: \"kubernetes.io/projected/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-kube-api-access-f5rjt\") pod \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\" (UID: \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\") "
Jul 24 19:20:13.340765 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.340665    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwrcr\" (UniqueName: \"kubernetes.io/projected/9d14eed6-d34f-49c2-bdfb-0932e02c526f-kube-api-access-nwrcr\") pod \"9d14eed6-d34f-49c2-bdfb-0932e02c526f\" (UID: \"9d14eed6-d34f-49c2-bdfb-0932e02c526f\") "
Jul 24 19:20:13.340765 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.340717    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-openshift-service-ca\") pod \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\" (UID: \"72d2bb9c-1735-49fc-9ca1-27645fa5cea1\") "
Jul 24 19:20:13.340934 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.340782    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d14eed6-d34f-49c2-bdfb-0932e02c526f-plugins-config-volume\") pod \"9d14eed6-d34f-49c2-bdfb-0932e02c526f\" (UID: \"9d14eed6-d34f-49c2-bdfb-0932e02c526f\") "
Jul 24 19:20:13.341408 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.341365    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d14eed6-d34f-49c2-bdfb-0932e02c526f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9d14eed6-d34f-49c2-bdfb-0932e02c526f" (UID: "9d14eed6-d34f-49c2-bdfb-0932e02c526f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:20:13.342302 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.341785    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "72d2bb9c-1735-49fc-9ca1-27645fa5cea1" (UID: "72d2bb9c-1735-49fc-9ca1-27645fa5cea1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:20:13.343945 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.343916    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d14eed6-d34f-49c2-bdfb-0932e02c526f-kube-api-access-nwrcr" (OuterVolumeSpecName: "kube-api-access-nwrcr") pod "9d14eed6-d34f-49c2-bdfb-0932e02c526f" (UID: "9d14eed6-d34f-49c2-bdfb-0932e02c526f"). InnerVolumeSpecName "kube-api-access-nwrcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:20:13.344074 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.343920    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "72d2bb9c-1735-49fc-9ca1-27645fa5cea1" (UID: "72d2bb9c-1735-49fc-9ca1-27645fa5cea1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:20:13.344360 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.344338    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-kube-api-access-f5rjt" (OuterVolumeSpecName: "kube-api-access-f5rjt") pod "72d2bb9c-1735-49fc-9ca1-27645fa5cea1" (UID: "72d2bb9c-1735-49fc-9ca1-27645fa5cea1"). InnerVolumeSpecName "kube-api-access-f5rjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:20:13.441809 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.441772    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j92pj\" (UniqueName: \"kubernetes.io/projected/0dd99a6f-a52b-439c-8c2c-1fc298a13b42-kube-api-access-j92pj\") pod \"0dd99a6f-a52b-439c-8c2c-1fc298a13b42\" (UID: \"0dd99a6f-a52b-439c-8c2c-1fc298a13b42\") "
Jul 24 19:20:13.441973 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.441856    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dd99a6f-a52b-439c-8c2c-1fc298a13b42-plugins-config-volume\") pod \"0dd99a6f-a52b-439c-8c2c-1fc298a13b42\" (UID: \"0dd99a6f-a52b-439c-8c2c-1fc298a13b42\") "
Jul 24 19:20:13.442016 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.441985    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d14eed6-d34f-49c2-bdfb-0932e02c526f-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:13.442016 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.441997    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:13.442016 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.442006    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5rjt\" (UniqueName: \"kubernetes.io/projected/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-kube-api-access-f5rjt\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:13.442016 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.442014    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwrcr\" (UniqueName: \"kubernetes.io/projected/9d14eed6-d34f-49c2-bdfb-0932e02c526f-kube-api-access-nwrcr\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:13.442143 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.442023    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d2bb9c-1735-49fc-9ca1-27645fa5cea1-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:13.442204 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.442181    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dd99a6f-a52b-439c-8c2c-1fc298a13b42-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0dd99a6f-a52b-439c-8c2c-1fc298a13b42" (UID: "0dd99a6f-a52b-439c-8c2c-1fc298a13b42"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:20:13.444133 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.444113    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dd99a6f-a52b-439c-8c2c-1fc298a13b42-kube-api-access-j92pj" (OuterVolumeSpecName: "kube-api-access-j92pj") pod "0dd99a6f-a52b-439c-8c2c-1fc298a13b42" (UID: "0dd99a6f-a52b-439c-8c2c-1fc298a13b42"). InnerVolumeSpecName "kube-api-access-j92pj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:20:13.494947 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.494867    2563 generic.go:356] "Generic (PLEG): container finished" podID="0dd99a6f-a52b-439c-8c2c-1fc298a13b42" containerID="eb5095f07e64874eb683b61bf50294750392826f6b5698f0c68eb0e0f0d21dce" exitCode=0
Jul 24 19:20:13.494947 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.494921    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"
Jul 24 19:20:13.495163 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.494957    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw" event={"ID":"0dd99a6f-a52b-439c-8c2c-1fc298a13b42","Type":"ContainerDied","Data":"eb5095f07e64874eb683b61bf50294750392826f6b5698f0c68eb0e0f0d21dce"}
Jul 24 19:20:13.495163 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.495045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw" event={"ID":"0dd99a6f-a52b-439c-8c2c-1fc298a13b42","Type":"ContainerDied","Data":"2038fdcb077ab3f6bf94111db49a441ae5d3ae3f2fbba18221c0e380d512915a"}
Jul 24 19:20:13.495163 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.495082    2563 scope.go:117] "RemoveContainer" containerID="eb5095f07e64874eb683b61bf50294750392826f6b5698f0c68eb0e0f0d21dce"
Jul 24 19:20:13.496517 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.496496    2563 generic.go:356] "Generic (PLEG): container finished" podID="9d14eed6-d34f-49c2-bdfb-0932e02c526f" containerID="d8b4dd3b3e805a528d46a49bd48faeadb1a4b2ec2a0e87e98b45ef4a48e4ccb7" exitCode=0
Jul 24 19:20:13.496619 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.496553    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"
Jul 24 19:20:13.496619 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.496589    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk" event={"ID":"9d14eed6-d34f-49c2-bdfb-0932e02c526f","Type":"ContainerDied","Data":"d8b4dd3b3e805a528d46a49bd48faeadb1a4b2ec2a0e87e98b45ef4a48e4ccb7"}
Jul 24 19:20:13.496739 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.496653    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk" event={"ID":"9d14eed6-d34f-49c2-bdfb-0932e02c526f","Type":"ContainerDied","Data":"624e1becc41525dad3e51372fde9e96b8d8ccf32435d4dc344a89f745b35cea3"}
Jul 24 19:20:13.497962 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.497938    2563 generic.go:356] "Generic (PLEG): container finished" podID="72d2bb9c-1735-49fc-9ca1-27645fa5cea1" containerID="37af2312ab5158857efb0b9257df32c88ca17faf7833281f7ec68f1603272dc3" exitCode=0
Jul 24 19:20:13.498055 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.497988    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp" event={"ID":"72d2bb9c-1735-49fc-9ca1-27645fa5cea1","Type":"ContainerDied","Data":"37af2312ab5158857efb0b9257df32c88ca17faf7833281f7ec68f1603272dc3"}
Jul 24 19:20:13.498055 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.498023    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"
Jul 24 19:20:13.498055 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.498030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp" event={"ID":"72d2bb9c-1735-49fc-9ca1-27645fa5cea1","Type":"ContainerDied","Data":"e47aa231ca3ce81fcd42bc41373adfc871709ce92e6e7a99c3d32b430761bea0"}
Jul 24 19:20:13.506014 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.506000    2563 scope.go:117] "RemoveContainer" containerID="eb5095f07e64874eb683b61bf50294750392826f6b5698f0c68eb0e0f0d21dce"
Jul 24 19:20:13.506336 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:20:13.506310    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb5095f07e64874eb683b61bf50294750392826f6b5698f0c68eb0e0f0d21dce\": container with ID starting with eb5095f07e64874eb683b61bf50294750392826f6b5698f0c68eb0e0f0d21dce not found: ID does not exist" containerID="eb5095f07e64874eb683b61bf50294750392826f6b5698f0c68eb0e0f0d21dce"
Jul 24 19:20:13.506434 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.506349    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb5095f07e64874eb683b61bf50294750392826f6b5698f0c68eb0e0f0d21dce"} err="failed to get container status \"eb5095f07e64874eb683b61bf50294750392826f6b5698f0c68eb0e0f0d21dce\": rpc error: code = NotFound desc = could not find container \"eb5095f07e64874eb683b61bf50294750392826f6b5698f0c68eb0e0f0d21dce\": container with ID starting with eb5095f07e64874eb683b61bf50294750392826f6b5698f0c68eb0e0f0d21dce not found: ID does not exist"
Jul 24 19:20:13.506434 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.506372    2563 scope.go:117] "RemoveContainer" containerID="d8b4dd3b3e805a528d46a49bd48faeadb1a4b2ec2a0e87e98b45ef4a48e4ccb7"
Jul 24 19:20:13.518708 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.518686    2563 scope.go:117] "RemoveContainer" containerID="d8b4dd3b3e805a528d46a49bd48faeadb1a4b2ec2a0e87e98b45ef4a48e4ccb7"
Jul 24 19:20:13.519073 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:20:13.519051    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8b4dd3b3e805a528d46a49bd48faeadb1a4b2ec2a0e87e98b45ef4a48e4ccb7\": container with ID starting with d8b4dd3b3e805a528d46a49bd48faeadb1a4b2ec2a0e87e98b45ef4a48e4ccb7 not found: ID does not exist" containerID="d8b4dd3b3e805a528d46a49bd48faeadb1a4b2ec2a0e87e98b45ef4a48e4ccb7"
Jul 24 19:20:13.519164 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.519084    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8b4dd3b3e805a528d46a49bd48faeadb1a4b2ec2a0e87e98b45ef4a48e4ccb7"} err="failed to get container status \"d8b4dd3b3e805a528d46a49bd48faeadb1a4b2ec2a0e87e98b45ef4a48e4ccb7\": rpc error: code = NotFound desc = could not find container \"d8b4dd3b3e805a528d46a49bd48faeadb1a4b2ec2a0e87e98b45ef4a48e4ccb7\": container with ID starting with d8b4dd3b3e805a528d46a49bd48faeadb1a4b2ec2a0e87e98b45ef4a48e4ccb7 not found: ID does not exist"
Jul 24 19:20:13.519164 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.519106    2563 scope.go:117] "RemoveContainer" containerID="37af2312ab5158857efb0b9257df32c88ca17faf7833281f7ec68f1603272dc3"
Jul 24 19:20:13.520105 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.520020    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"]
Jul 24 19:20:13.521985 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.521966    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-950cf0-6b5fbcd464-qmnnw"]
Jul 24 19:20:13.526776 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.526762    2563 scope.go:117] "RemoveContainer" containerID="37af2312ab5158857efb0b9257df32c88ca17faf7833281f7ec68f1603272dc3"
Jul 24 19:20:13.527020 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:20:13.527003    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37af2312ab5158857efb0b9257df32c88ca17faf7833281f7ec68f1603272dc3\": container with ID starting with 37af2312ab5158857efb0b9257df32c88ca17faf7833281f7ec68f1603272dc3 not found: ID does not exist" containerID="37af2312ab5158857efb0b9257df32c88ca17faf7833281f7ec68f1603272dc3"
Jul 24 19:20:13.527068 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.527034    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37af2312ab5158857efb0b9257df32c88ca17faf7833281f7ec68f1603272dc3"} err="failed to get container status \"37af2312ab5158857efb0b9257df32c88ca17faf7833281f7ec68f1603272dc3\": rpc error: code = NotFound desc = could not find container \"37af2312ab5158857efb0b9257df32c88ca17faf7833281f7ec68f1603272dc3\": container with ID starting with 37af2312ab5158857efb0b9257df32c88ca17faf7833281f7ec68f1603272dc3 not found: ID does not exist"
Jul 24 19:20:13.530516 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.530488    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"]
Jul 24 19:20:13.532483 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.532458    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-950cf0-56c5586956-5dwtk"]
Jul 24 19:20:13.542179 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.542156    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"]
Jul 24 19:20:13.542513 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.542498    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j92pj\" (UniqueName: \"kubernetes.io/projected/0dd99a6f-a52b-439c-8c2c-1fc298a13b42-kube-api-access-j92pj\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:13.542597 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.542516    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dd99a6f-a52b-439c-8c2c-1fc298a13b42-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:13.548161 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:13.548144    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-950cf0-569d995975-4xffp"]
Jul 24 19:20:14.203866 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:14.203835    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dd99a6f-a52b-439c-8c2c-1fc298a13b42" path="/var/lib/kubelet/pods/0dd99a6f-a52b-439c-8c2c-1fc298a13b42/volumes"
Jul 24 19:20:14.204228 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:14.204192    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72d2bb9c-1735-49fc-9ca1-27645fa5cea1" path="/var/lib/kubelet/pods/72d2bb9c-1735-49fc-9ca1-27645fa5cea1/volumes"
Jul 24 19:20:14.204508 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:14.204494    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d14eed6-d34f-49c2-bdfb-0932e02c526f" path="/var/lib/kubelet/pods/9d14eed6-d34f-49c2-bdfb-0932e02c526f/volumes"
Jul 24 19:20:21.655224 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:21.655184    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:20:24.159876 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.159839    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"]
Jul 24 19:20:24.160334 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.160154    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl" podUID="7594d561-a553-4388-b2dd-5b4709e05008" containerName="istio-proxy" containerID="cri-o://2c449d689f78c4fa34bf61357bfbb673cda892ade72d7326fa99ff9ec5ee5062" gracePeriod=30
Jul 24 19:20:24.500729 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.500637    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl"]
Jul 24 19:20:24.501054 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.501034    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d14eed6-d34f-49c2-bdfb-0932e02c526f" containerName="payload-processing"
Jul 24 19:20:24.501145 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.501058    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d14eed6-d34f-49c2-bdfb-0932e02c526f" containerName="payload-processing"
Jul 24 19:20:24.501145 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.501088    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72d2bb9c-1735-49fc-9ca1-27645fa5cea1" containerName="maas-api"
Jul 24 19:20:24.501145 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.501097    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="72d2bb9c-1735-49fc-9ca1-27645fa5cea1" containerName="maas-api"
Jul 24 19:20:24.501145 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.501116    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dd99a6f-a52b-439c-8c2c-1fc298a13b42" containerName="payload-pre-processing"
Jul 24 19:20:24.501145 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.501126    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dd99a6f-a52b-439c-8c2c-1fc298a13b42" containerName="payload-pre-processing"
Jul 24 19:20:24.501431 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.501205    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dd99a6f-a52b-439c-8c2c-1fc298a13b42" containerName="payload-pre-processing"
Jul 24 19:20:24.501431 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.501218    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d14eed6-d34f-49c2-bdfb-0932e02c526f" containerName="payload-processing"
Jul 24 19:20:24.501431 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.501229    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="72d2bb9c-1735-49fc-9ca1-27645fa5cea1" containerName="maas-api"
Jul 24 19:20:24.503985 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.503965    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl"
Jul 24 19:20:24.506292 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.506267    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6gvkz\""
Jul 24 19:20:24.511336 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.511301    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl"]
Jul 24 19:20:24.538951 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.538919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2979a45-7290-49e2-a687-2f2a37bbc533-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-948d66-87msl\" (UID: \"d2979a45-7290-49e2-a687-2f2a37bbc533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl"
Jul 24 19:20:24.640376 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.640331    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2979a45-7290-49e2-a687-2f2a37bbc533-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-948d66-87msl\" (UID: \"d2979a45-7290-49e2-a687-2f2a37bbc533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl"
Jul 24 19:20:24.641064 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.641041    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2979a45-7290-49e2-a687-2f2a37bbc533-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-948d66-87msl\" (UID: \"d2979a45-7290-49e2-a687-2f2a37bbc533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl"
Jul 24 19:20:24.815602 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.815562    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl"
Jul 24 19:20:24.948911 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:24.948888    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl"]
Jul 24 19:20:24.951211 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:20:24.951183    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2979a45_7290_49e2_a687_2f2a37bbc533.slice/crio-631905b3914f1d0a36230cc1d53ed123906f408bb251511de1774cc34e531650 WatchSource:0}: Error finding container 631905b3914f1d0a36230cc1d53ed123906f408bb251511de1774cc34e531650: Status 404 returned error can't find the container with id 631905b3914f1d0a36230cc1d53ed123906f408bb251511de1774cc34e531650
Jul 24 19:20:25.562826 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:25.562780    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl" event={"ID":"d2979a45-7290-49e2-a687-2f2a37bbc533","Type":"ContainerStarted","Data":"631905b3914f1d0a36230cc1d53ed123906f408bb251511de1774cc34e531650"}
Jul 24 19:20:27.575113 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:27.575069    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl" event={"ID":"d2979a45-7290-49e2-a687-2f2a37bbc533","Type":"ContainerStarted","Data":"02e96e30f1ef1f4b030b1534177e550e2090abfd062a5fa7ef812359828bac5d"}
Jul 24 19:20:27.591021 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:27.590975    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl" podStartSLOduration=1.283530763 podStartE2EDuration="3.590960912s" totalImagesPullingTime="2.307430149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:20:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:20:27.590173318 +0000 UTC m=+2165.981124796" watchObservedRunningTime="2026-07-24 19:20:27.590960912 +0000 UTC m=+2165.981912412"
Jul 24 19:20:28.450748 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:28.450702    2563 patch_prober.go:28] interesting pod/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 19:20:28.450925 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:28.450761    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl" podUID="7594d561-a553-4388-b2dd-5b4709e05008" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 19:20:28.584379 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:28.584346    2563 generic.go:356] "Generic (PLEG): container finished" podID="d2979a45-7290-49e2-a687-2f2a37bbc533" containerID="02e96e30f1ef1f4b030b1534177e550e2090abfd062a5fa7ef812359828bac5d" exitCode=0
Jul 24 19:20:28.584794 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:28.584439    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl" event={"ID":"d2979a45-7290-49e2-a687-2f2a37bbc533","Type":"ContainerDied","Data":"02e96e30f1ef1f4b030b1534177e550e2090abfd062a5fa7ef812359828bac5d"}
Jul 24 19:20:29.412755 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.412730    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:20:29.485572 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.485487    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-istio-envoy\") pod \"7594d561-a553-4388-b2dd-5b4709e05008\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") "
Jul 24 19:20:29.485572 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.485523    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-workload-certs\") pod \"7594d561-a553-4388-b2dd-5b4709e05008\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") "
Jul 24 19:20:29.485803 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.485589    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-credential-socket\") pod \"7594d561-a553-4388-b2dd-5b4709e05008\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") "
Jul 24 19:20:29.485803 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.485619    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-istio-data\") pod \"7594d561-a553-4388-b2dd-5b4709e05008\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") "
Jul 24 19:20:29.485803 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.485666    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7594d561-a553-4388-b2dd-5b4709e05008-istio-podinfo\") pod \"7594d561-a553-4388-b2dd-5b4709e05008\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") "
Jul 24 19:20:29.485803 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.485707    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhp2j\" (UniqueName: \"kubernetes.io/projected/7594d561-a553-4388-b2dd-5b4709e05008-kube-api-access-xhp2j\") pod \"7594d561-a553-4388-b2dd-5b4709e05008\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") "
Jul 24 19:20:29.485803 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.485753    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7594d561-a553-4388-b2dd-5b4709e05008-istio-token\") pod \"7594d561-a553-4388-b2dd-5b4709e05008\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") "
Jul 24 19:20:29.485803 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.485793    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-workload-socket\") pod \"7594d561-a553-4388-b2dd-5b4709e05008\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") "
Jul 24 19:20:29.486119 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.485820    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7594d561-a553-4388-b2dd-5b4709e05008-istiod-ca-cert\") pod \"7594d561-a553-4388-b2dd-5b4709e05008\" (UID: \"7594d561-a553-4388-b2dd-5b4709e05008\") "
Jul 24 19:20:29.486119 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.485851    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7594d561-a553-4388-b2dd-5b4709e05008" (UID: "7594d561-a553-4388-b2dd-5b4709e05008"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:20:29.486119 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.485922    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7594d561-a553-4388-b2dd-5b4709e05008" (UID: "7594d561-a553-4388-b2dd-5b4709e05008"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:20:29.486119 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.486077    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-workload-certs\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:29.486119 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.486096    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-credential-socket\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:29.486412 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.486216    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7594d561-a553-4388-b2dd-5b4709e05008-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7594d561-a553-4388-b2dd-5b4709e05008" (UID: "7594d561-a553-4388-b2dd-5b4709e05008"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:20:29.486685 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.486661    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7594d561-a553-4388-b2dd-5b4709e05008" (UID: "7594d561-a553-4388-b2dd-5b4709e05008"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:20:29.486948 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.486917    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-istio-data" (OuterVolumeSpecName: "istio-data") pod "7594d561-a553-4388-b2dd-5b4709e05008" (UID: "7594d561-a553-4388-b2dd-5b4709e05008"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:20:29.488130 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.488100    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7594d561-a553-4388-b2dd-5b4709e05008" (UID: "7594d561-a553-4388-b2dd-5b4709e05008"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:20:29.488304 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.488281    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7594d561-a553-4388-b2dd-5b4709e05008-istio-token" (OuterVolumeSpecName: "istio-token") pod "7594d561-a553-4388-b2dd-5b4709e05008" (UID: "7594d561-a553-4388-b2dd-5b4709e05008"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:20:29.488364 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.488298    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7594d561-a553-4388-b2dd-5b4709e05008-kube-api-access-xhp2j" (OuterVolumeSpecName: "kube-api-access-xhp2j") pod "7594d561-a553-4388-b2dd-5b4709e05008" (UID: "7594d561-a553-4388-b2dd-5b4709e05008"). InnerVolumeSpecName "kube-api-access-xhp2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:20:29.488364 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.488343    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7594d561-a553-4388-b2dd-5b4709e05008-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7594d561-a553-4388-b2dd-5b4709e05008" (UID: "7594d561-a553-4388-b2dd-5b4709e05008"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 19:20:29.586800 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.586770    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7594d561-a553-4388-b2dd-5b4709e05008-istio-token\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:29.586800 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.586802    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-workload-socket\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:29.587218 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.586813    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7594d561-a553-4388-b2dd-5b4709e05008-istiod-ca-cert\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:29.587218 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.586822    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-istio-envoy\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:29.587218 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.586830    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7594d561-a553-4388-b2dd-5b4709e05008-istio-data\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:29.587218 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.586837    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7594d561-a553-4388-b2dd-5b4709e05008-istio-podinfo\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:29.587218 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.586845    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhp2j\" (UniqueName: \"kubernetes.io/projected/7594d561-a553-4388-b2dd-5b4709e05008-kube-api-access-xhp2j\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:29.591102 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.591076    2563 generic.go:356] "Generic (PLEG): container finished" podID="7594d561-a553-4388-b2dd-5b4709e05008" containerID="2c449d689f78c4fa34bf61357bfbb673cda892ade72d7326fa99ff9ec5ee5062" exitCode=0
Jul 24 19:20:29.591250 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.591132    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"
Jul 24 19:20:29.591250 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.591161    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl" event={"ID":"7594d561-a553-4388-b2dd-5b4709e05008","Type":"ContainerDied","Data":"2c449d689f78c4fa34bf61357bfbb673cda892ade72d7326fa99ff9ec5ee5062"}
Jul 24 19:20:29.591250 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.591238    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl" event={"ID":"7594d561-a553-4388-b2dd-5b4709e05008","Type":"ContainerDied","Data":"074dd521d13c86359deefb0b9df460f1e168682310e88d36fb683c9b4a70ffb7"}
Jul 24 19:20:29.591410 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.591267    2563 scope.go:117] "RemoveContainer" containerID="2c449d689f78c4fa34bf61357bfbb673cda892ade72d7326fa99ff9ec5ee5062"
Jul 24 19:20:29.601305 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.601286    2563 scope.go:117] "RemoveContainer" containerID="499bd3e14dfd024ddd700da74bc0980468ac1c354966462a54413a8c32d010ee"
Jul 24 19:20:29.610595 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.610573    2563 scope.go:117] "RemoveContainer" containerID="2c449d689f78c4fa34bf61357bfbb673cda892ade72d7326fa99ff9ec5ee5062"
Jul 24 19:20:29.610906 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:20:29.610886    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c449d689f78c4fa34bf61357bfbb673cda892ade72d7326fa99ff9ec5ee5062\": container with ID starting with 2c449d689f78c4fa34bf61357bfbb673cda892ade72d7326fa99ff9ec5ee5062 not found: ID does not exist" containerID="2c449d689f78c4fa34bf61357bfbb673cda892ade72d7326fa99ff9ec5ee5062"
Jul 24 19:20:29.610987 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.610915    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c449d689f78c4fa34bf61357bfbb673cda892ade72d7326fa99ff9ec5ee5062"} err="failed to get container status \"2c449d689f78c4fa34bf61357bfbb673cda892ade72d7326fa99ff9ec5ee5062\": rpc error: code = NotFound desc = could not find container \"2c449d689f78c4fa34bf61357bfbb673cda892ade72d7326fa99ff9ec5ee5062\": container with ID starting with 2c449d689f78c4fa34bf61357bfbb673cda892ade72d7326fa99ff9ec5ee5062 not found: ID does not exist"
Jul 24 19:20:29.610987 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.610935    2563 scope.go:117] "RemoveContainer" containerID="499bd3e14dfd024ddd700da74bc0980468ac1c354966462a54413a8c32d010ee"
Jul 24 19:20:29.611210 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:20:29.611192    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"499bd3e14dfd024ddd700da74bc0980468ac1c354966462a54413a8c32d010ee\": container with ID starting with 499bd3e14dfd024ddd700da74bc0980468ac1c354966462a54413a8c32d010ee not found: ID does not exist" containerID="499bd3e14dfd024ddd700da74bc0980468ac1c354966462a54413a8c32d010ee"
Jul 24 19:20:29.611253 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.611216    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"499bd3e14dfd024ddd700da74bc0980468ac1c354966462a54413a8c32d010ee"} err="failed to get container status \"499bd3e14dfd024ddd700da74bc0980468ac1c354966462a54413a8c32d010ee\": rpc error: code = NotFound desc = could not find container \"499bd3e14dfd024ddd700da74bc0980468ac1c354966462a54413a8c32d010ee\": container with ID starting with 499bd3e14dfd024ddd700da74bc0980468ac1c354966462a54413a8c32d010ee not found: ID does not exist"
Jul 24 19:20:29.617519 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.617495    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"]
Jul 24 19:20:29.621705 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.621679    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-950cf0-openshift-default-655876b6dd-6rsxl"]
Jul 24 19:20:29.732948 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.732923    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl"
Jul 24 19:20:29.788475 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.788321    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2979a45-7290-49e2-a687-2f2a37bbc533-maas-api-service-ca\") pod \"d2979a45-7290-49e2-a687-2f2a37bbc533\" (UID: \"d2979a45-7290-49e2-a687-2f2a37bbc533\") "
Jul 24 19:20:29.788787 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.788759    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2979a45-7290-49e2-a687-2f2a37bbc533-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d2979a45-7290-49e2-a687-2f2a37bbc533" (UID: "d2979a45-7290-49e2-a687-2f2a37bbc533"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:20:29.889021 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:29.888986    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2979a45-7290-49e2-a687-2f2a37bbc533-maas-api-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:30.204181 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:30.204146    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7594d561-a553-4388-b2dd-5b4709e05008" path="/var/lib/kubelet/pods/7594d561-a553-4388-b2dd-5b4709e05008/volumes"
Jul 24 19:20:30.597113 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:30.597084    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl"
Jul 24 19:20:30.597519 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:30.597123    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl" event={"ID":"d2979a45-7290-49e2-a687-2f2a37bbc533","Type":"ContainerDied","Data":"631905b3914f1d0a36230cc1d53ed123906f408bb251511de1774cc34e531650"}
Jul 24 19:20:30.597519 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:30.597168    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="631905b3914f1d0a36230cc1d53ed123906f408bb251511de1774cc34e531650"
Jul 24 19:20:34.934407 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:34.934186    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"]
Jul 24 19:20:34.934895 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:34.934564    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq" podUID="23f99621-316f-46cd-98fd-7533dc9e2ca2" containerName="maas-api" containerID="cri-o://925f09dc20427144ca3b11cd932701dc5f48841a8654eb327c48f185ac9e7046" gracePeriod=30
Jul 24 19:20:34.994125 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:34.994078    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq" podUID="23f99621-316f-46cd-98fd-7533dc9e2ca2" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.52:8443/health\": dial tcp 10.134.0.52:8443: connect: connection refused"
Jul 24 19:20:35.178785 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.178758    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:20:35.337986 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.337959    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23f99621-316f-46cd-98fd-7533dc9e2ca2-openshift-service-ca\") pod \"23f99621-316f-46cd-98fd-7533dc9e2ca2\" (UID: \"23f99621-316f-46cd-98fd-7533dc9e2ca2\") "
Jul 24 19:20:35.338181 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.338043    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgf7j\" (UniqueName: \"kubernetes.io/projected/23f99621-316f-46cd-98fd-7533dc9e2ca2-kube-api-access-jgf7j\") pod \"23f99621-316f-46cd-98fd-7533dc9e2ca2\" (UID: \"23f99621-316f-46cd-98fd-7533dc9e2ca2\") "
Jul 24 19:20:35.338181 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.338081    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23f99621-316f-46cd-98fd-7533dc9e2ca2-maas-api-tls\") pod \"23f99621-316f-46cd-98fd-7533dc9e2ca2\" (UID: \"23f99621-316f-46cd-98fd-7533dc9e2ca2\") "
Jul 24 19:20:35.338374 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.338352    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23f99621-316f-46cd-98fd-7533dc9e2ca2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "23f99621-316f-46cd-98fd-7533dc9e2ca2" (UID: "23f99621-316f-46cd-98fd-7533dc9e2ca2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:20:35.340304 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.340274    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23f99621-316f-46cd-98fd-7533dc9e2ca2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "23f99621-316f-46cd-98fd-7533dc9e2ca2" (UID: "23f99621-316f-46cd-98fd-7533dc9e2ca2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:20:35.340543 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.340522    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23f99621-316f-46cd-98fd-7533dc9e2ca2-kube-api-access-jgf7j" (OuterVolumeSpecName: "kube-api-access-jgf7j") pod "23f99621-316f-46cd-98fd-7533dc9e2ca2" (UID: "23f99621-316f-46cd-98fd-7533dc9e2ca2"). InnerVolumeSpecName "kube-api-access-jgf7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:20:35.439420 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.439351    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgf7j\" (UniqueName: \"kubernetes.io/projected/23f99621-316f-46cd-98fd-7533dc9e2ca2-kube-api-access-jgf7j\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:35.439420 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.439416    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23f99621-316f-46cd-98fd-7533dc9e2ca2-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:35.439420 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.439428    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23f99621-316f-46cd-98fd-7533dc9e2ca2-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:20:35.622273 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.622179    2563 generic.go:356] "Generic (PLEG): container finished" podID="23f99621-316f-46cd-98fd-7533dc9e2ca2" containerID="925f09dc20427144ca3b11cd932701dc5f48841a8654eb327c48f185ac9e7046" exitCode=0
Jul 24 19:20:35.622273 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.622238    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"
Jul 24 19:20:35.622539 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.622241    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq" event={"ID":"23f99621-316f-46cd-98fd-7533dc9e2ca2","Type":"ContainerDied","Data":"925f09dc20427144ca3b11cd932701dc5f48841a8654eb327c48f185ac9e7046"}
Jul 24 19:20:35.622539 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.622385    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq" event={"ID":"23f99621-316f-46cd-98fd-7533dc9e2ca2","Type":"ContainerDied","Data":"4468e22df1925c7bb6fb961cfb2c74f69d4ec10358689e962023c0a598daa5bd"}
Jul 24 19:20:35.622539 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.622430    2563 scope.go:117] "RemoveContainer" containerID="925f09dc20427144ca3b11cd932701dc5f48841a8654eb327c48f185ac9e7046"
Jul 24 19:20:35.632002 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.631984    2563 scope.go:117] "RemoveContainer" containerID="925f09dc20427144ca3b11cd932701dc5f48841a8654eb327c48f185ac9e7046"
Jul 24 19:20:35.632229 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:20:35.632212    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"925f09dc20427144ca3b11cd932701dc5f48841a8654eb327c48f185ac9e7046\": container with ID starting with 925f09dc20427144ca3b11cd932701dc5f48841a8654eb327c48f185ac9e7046 not found: ID does not exist" containerID="925f09dc20427144ca3b11cd932701dc5f48841a8654eb327c48f185ac9e7046"
Jul 24 19:20:35.632281 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.632238    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"925f09dc20427144ca3b11cd932701dc5f48841a8654eb327c48f185ac9e7046"} err="failed to get container status \"925f09dc20427144ca3b11cd932701dc5f48841a8654eb327c48f185ac9e7046\": rpc error: code = NotFound desc = could not find container \"925f09dc20427144ca3b11cd932701dc5f48841a8654eb327c48f185ac9e7046\": container with ID starting with 925f09dc20427144ca3b11cd932701dc5f48841a8654eb327c48f185ac9e7046 not found: ID does not exist"
Jul 24 19:20:35.644161 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.644108    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"]
Jul 24 19:20:35.646046 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:35.646022    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-948d66-59555cbffb-bwdzq"]
Jul 24 19:20:36.204861 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:36.204829    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23f99621-316f-46cd-98fd-7533dc9e2ca2" path="/var/lib/kubelet/pods/23f99621-316f-46cd-98fd-7533dc9e2ca2/volumes"
Jul 24 19:20:37.843301 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:37.843264    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:20:39.577975 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:39.577937    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl"]
Jul 24 19:20:39.581857 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:39.581832    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-948d66-87msl"]
Jul 24 19:20:40.203615 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:40.203581    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2979a45-7290-49e2-a687-2f2a37bbc533" path="/var/lib/kubelet/pods/d2979a45-7290-49e2-a687-2f2a37bbc533/volumes"
Jul 24 19:20:47.316833 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.316797    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"]
Jul 24 19:20:47.317236 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.317101    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7594d561-a553-4388-b2dd-5b4709e05008" containerName="istio-proxy"
Jul 24 19:20:47.317236 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.317111    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7594d561-a553-4388-b2dd-5b4709e05008" containerName="istio-proxy"
Jul 24 19:20:47.317236 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.317127    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23f99621-316f-46cd-98fd-7533dc9e2ca2" containerName="maas-api"
Jul 24 19:20:47.317236 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.317133    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="23f99621-316f-46cd-98fd-7533dc9e2ca2" containerName="maas-api"
Jul 24 19:20:47.317236 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.317143    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2979a45-7290-49e2-a687-2f2a37bbc533" containerName="revoke-keys"
Jul 24 19:20:47.317236 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.317148    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2979a45-7290-49e2-a687-2f2a37bbc533" containerName="revoke-keys"
Jul 24 19:20:47.317236 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.317216    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7594d561-a553-4388-b2dd-5b4709e05008" containerName="istio-proxy"
Jul 24 19:20:47.317236 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.317224    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2979a45-7290-49e2-a687-2f2a37bbc533" containerName="revoke-keys"
Jul 24 19:20:47.317236 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.317231    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="23f99621-316f-46cd-98fd-7533dc9e2ca2" containerName="maas-api"
Jul 24 19:20:47.321326 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.321309    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.324677 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.324659    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-4bd2a2-openshift-default-dockercfg-vbjdt\""
Jul 24 19:20:47.335097 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.335069    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"]
Jul 24 19:20:47.446856 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.446823    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-istio-data\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.446856 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.446865    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99745325-561f-4a1d-afbe-bdb95274757f-istio-podinfo\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.447111 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.446887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-istio-envoy\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.447111 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.446913    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-workload-certs\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.447111 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.447024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-workload-socket\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.447111 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.447052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-257pv\" (UniqueName: \"kubernetes.io/projected/99745325-561f-4a1d-afbe-bdb95274757f-kube-api-access-257pv\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.447111 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.447083    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99745325-561f-4a1d-afbe-bdb95274757f-istio-token\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.447111 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.447109    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-credential-socket\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.447369 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.447132    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99745325-561f-4a1d-afbe-bdb95274757f-istiod-ca-cert\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.548020 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.547985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-workload-socket\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.548020 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.548020    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-257pv\" (UniqueName: \"kubernetes.io/projected/99745325-561f-4a1d-afbe-bdb95274757f-kube-api-access-257pv\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.548273 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.548044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99745325-561f-4a1d-afbe-bdb95274757f-istio-token\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.548273 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.548061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-credential-socket\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.548273 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.548092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99745325-561f-4a1d-afbe-bdb95274757f-istiod-ca-cert\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.548273 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.548122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-istio-data\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.548273 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.548155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99745325-561f-4a1d-afbe-bdb95274757f-istio-podinfo\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.548273 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.548182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-istio-envoy\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.548273 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.548212    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-workload-certs\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.548644 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.548439    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-workload-socket\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.548644 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.548553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-credential-socket\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.548644 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.548564    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-workload-certs\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.548644 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.548633    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-istio-data\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.548776 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.548764    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99745325-561f-4a1d-afbe-bdb95274757f-istiod-ca-cert\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.550607 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.550589    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-istio-envoy\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.550814 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.550789    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99745325-561f-4a1d-afbe-bdb95274757f-istio-podinfo\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.555946 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.555915    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99745325-561f-4a1d-afbe-bdb95274757f-istio-token\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.556055 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.556030    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-257pv\" (UniqueName: \"kubernetes.io/projected/99745325-561f-4a1d-afbe-bdb95274757f-kube-api-access-257pv\") pod \"e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") " pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.631591 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.631508    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:47.771773 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.771747    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"]
Jul 24 19:20:47.773813 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:20:47.773785    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99745325_561f_4a1d_afbe_bdb95274757f.slice/crio-456bcfc10a21fe658a874ed71abb7f019b8af1327b0993354d417f8075097bb6 WatchSource:0}: Error finding container 456bcfc10a21fe658a874ed71abb7f019b8af1327b0993354d417f8075097bb6: Status 404 returned error can't find the container with id 456bcfc10a21fe658a874ed71abb7f019b8af1327b0993354d417f8075097bb6
Jul 24 19:20:47.775900 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.775870    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:20:47.776001 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.775931    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:20:47.776001 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:47.775958    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:20:48.696083 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:48.696045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8" event={"ID":"99745325-561f-4a1d-afbe-bdb95274757f","Type":"ContainerStarted","Data":"f754b9a3c473e4717803f08458d9f33203627db02e6d8ea7a171f325d040f7f7"}
Jul 24 19:20:48.696561 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:48.696097    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8" event={"ID":"99745325-561f-4a1d-afbe-bdb95274757f","Type":"ContainerStarted","Data":"456bcfc10a21fe658a874ed71abb7f019b8af1327b0993354d417f8075097bb6"}
Jul 24 19:20:48.716629 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:48.716579    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8" podStartSLOduration=1.71656305 podStartE2EDuration="1.71656305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:20:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:20:48.714884968 +0000 UTC m=+2187.105836444" watchObservedRunningTime="2026-07-24 19:20:48.71656305 +0000 UTC m=+2187.107514526"
Jul 24 19:20:49.632098 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:49.632054    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:49.636941 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:49.636916    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:49.702353 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:49.702326    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:49.703232 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:49.703214    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:20:50.689935 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.689901    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"]
Jul 24 19:20:50.690237 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.690223    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7594d561-a553-4388-b2dd-5b4709e05008" containerName="istio-proxy"
Jul 24 19:20:50.690276 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.690239    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7594d561-a553-4388-b2dd-5b4709e05008" containerName="istio-proxy"
Jul 24 19:20:50.690345 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.690308    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7594d561-a553-4388-b2dd-5b4709e05008" containerName="istio-proxy"
Jul 24 19:20:50.693414 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.693383    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:20:50.695929 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.695909    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 19:20:50.696324 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.696310    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cvjqn\""
Jul 24 19:20:50.696528 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.696514    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-4bd2a2-serving-cert\""
Jul 24 19:20:50.696807 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.696794    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 19:20:50.704328 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.704305    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"]
Jul 24 19:20:50.774180 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.774148    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43533694-1e46-4364-a9d1-1ec037af912f-openshift-service-ca\") pod \"maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn\" (UID: \"43533694-1e46-4364-a9d1-1ec037af912f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:20:50.774372 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.774253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43533694-1e46-4364-a9d1-1ec037af912f-maas-api-tls\") pod \"maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn\" (UID: \"43533694-1e46-4364-a9d1-1ec037af912f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:20:50.774372 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.774295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9vmd\" (UniqueName: \"kubernetes.io/projected/43533694-1e46-4364-a9d1-1ec037af912f-kube-api-access-j9vmd\") pod \"maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn\" (UID: \"43533694-1e46-4364-a9d1-1ec037af912f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:20:50.875729 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.875697    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43533694-1e46-4364-a9d1-1ec037af912f-openshift-service-ca\") pod \"maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn\" (UID: \"43533694-1e46-4364-a9d1-1ec037af912f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:20:50.875917 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.875768    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43533694-1e46-4364-a9d1-1ec037af912f-maas-api-tls\") pod \"maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn\" (UID: \"43533694-1e46-4364-a9d1-1ec037af912f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:20:50.875917 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.875806    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9vmd\" (UniqueName: \"kubernetes.io/projected/43533694-1e46-4364-a9d1-1ec037af912f-kube-api-access-j9vmd\") pod \"maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn\" (UID: \"43533694-1e46-4364-a9d1-1ec037af912f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:20:50.876000 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:20:50.875914    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-serving-cert: secret "maas-api-e2e-api-a-4bd2a2-serving-cert" not found
Jul 24 19:20:50.876000 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:20:50.875990    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43533694-1e46-4364-a9d1-1ec037af912f-maas-api-tls podName:43533694-1e46-4364-a9d1-1ec037af912f nodeName:}" failed. No retries permitted until 2026-07-24 19:20:51.37597375 +0000 UTC m=+2189.766925205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/43533694-1e46-4364-a9d1-1ec037af912f-maas-api-tls") pod "maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn" (UID: "43533694-1e46-4364-a9d1-1ec037af912f") : secret "maas-api-e2e-api-a-4bd2a2-serving-cert" not found
Jul 24 19:20:50.876453 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.876437    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43533694-1e46-4364-a9d1-1ec037af912f-openshift-service-ca\") pod \"maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn\" (UID: \"43533694-1e46-4364-a9d1-1ec037af912f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:20:50.885322 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:50.885297    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9vmd\" (UniqueName: \"kubernetes.io/projected/43533694-1e46-4364-a9d1-1ec037af912f-kube-api-access-j9vmd\") pod \"maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn\" (UID: \"43533694-1e46-4364-a9d1-1ec037af912f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:20:51.380310 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.380277    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43533694-1e46-4364-a9d1-1ec037af912f-maas-api-tls\") pod \"maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn\" (UID: \"43533694-1e46-4364-a9d1-1ec037af912f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:20:51.382872 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.382853    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43533694-1e46-4364-a9d1-1ec037af912f-maas-api-tls\") pod \"maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn\" (UID: \"43533694-1e46-4364-a9d1-1ec037af912f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:20:51.455242 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.455203    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"]
Jul 24 19:20:51.458608 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.458590    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"
Jul 24 19:20:51.462589 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.462566    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-4bd2a2-dockercfg-d6r66\""
Jul 24 19:20:51.462716 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.462677    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-4bd2a2\""
Jul 24 19:20:51.476698 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.476673    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"]
Jul 24 19:20:51.480905 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.480877    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bea8eb12-e592-4345-b6ef-e5e68de925cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg\" (UID: \"bea8eb12-e592-4345-b6ef-e5e68de925cb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"
Jul 24 19:20:51.481011 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.480927    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d2fz\" (UniqueName: \"kubernetes.io/projected/bea8eb12-e592-4345-b6ef-e5e68de925cb-kube-api-access-5d2fz\") pod \"payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg\" (UID: \"bea8eb12-e592-4345-b6ef-e5e68de925cb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"
Jul 24 19:20:51.581826 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.581788    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bea8eb12-e592-4345-b6ef-e5e68de925cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg\" (UID: \"bea8eb12-e592-4345-b6ef-e5e68de925cb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"
Jul 24 19:20:51.581826 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.581830    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d2fz\" (UniqueName: \"kubernetes.io/projected/bea8eb12-e592-4345-b6ef-e5e68de925cb-kube-api-access-5d2fz\") pod \"payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg\" (UID: \"bea8eb12-e592-4345-b6ef-e5e68de925cb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"
Jul 24 19:20:51.582501 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.582484    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bea8eb12-e592-4345-b6ef-e5e68de925cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg\" (UID: \"bea8eb12-e592-4345-b6ef-e5e68de925cb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"
Jul 24 19:20:51.593547 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.593522    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d2fz\" (UniqueName: \"kubernetes.io/projected/bea8eb12-e592-4345-b6ef-e5e68de925cb-kube-api-access-5d2fz\") pod \"payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg\" (UID: \"bea8eb12-e592-4345-b6ef-e5e68de925cb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"
Jul 24 19:20:51.597953 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.597929    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"]
Jul 24 19:20:51.601723 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.601707    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"
Jul 24 19:20:51.604407 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.604378    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:20:51.613044 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.613013    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"]
Jul 24 19:20:51.682532 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.682498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b\" (UID: \"f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395\") " pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"
Jul 24 19:20:51.682775 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.682595    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzdd8\" (UniqueName: \"kubernetes.io/projected/f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395-kube-api-access-pzdd8\") pod \"payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b\" (UID: \"f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395\") " pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"
Jul 24 19:20:51.747186 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.747160    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"]
Jul 24 19:20:51.748491 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:20:51.748462    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43533694_1e46_4364_a9d1_1ec037af912f.slice/crio-b424a7cdf464765d0dfba4b30680757ad52fd638fa14e2cf8b9bd95e2d72a5bf WatchSource:0}: Error finding container b424a7cdf464765d0dfba4b30680757ad52fd638fa14e2cf8b9bd95e2d72a5bf: Status 404 returned error can't find the container with id b424a7cdf464765d0dfba4b30680757ad52fd638fa14e2cf8b9bd95e2d72a5bf
Jul 24 19:20:51.749850 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.749832    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 19:20:51.768303 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.768281    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"
Jul 24 19:20:51.784048 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.784017    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzdd8\" (UniqueName: \"kubernetes.io/projected/f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395-kube-api-access-pzdd8\") pod \"payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b\" (UID: \"f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395\") " pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"
Jul 24 19:20:51.784356 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.784330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b\" (UID: \"f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395\") " pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"
Jul 24 19:20:51.784955 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.784936    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b\" (UID: \"f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395\") " pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"
Jul 24 19:20:51.793200 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.793175    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzdd8\" (UniqueName: \"kubernetes.io/projected/f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395-kube-api-access-pzdd8\") pod \"payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b\" (UID: \"f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395\") " pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"
Jul 24 19:20:51.898493 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.898464    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"]
Jul 24 19:20:51.900794 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:20:51.900767    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbea8eb12_e592_4345_b6ef_e5e68de925cb.slice/crio-9188497cf0eb0fa03447533b5c5bd87fa3f0ebb35d3b1f7d5cea01126a84c51d WatchSource:0}: Error finding container 9188497cf0eb0fa03447533b5c5bd87fa3f0ebb35d3b1f7d5cea01126a84c51d: Status 404 returned error can't find the container with id 9188497cf0eb0fa03447533b5c5bd87fa3f0ebb35d3b1f7d5cea01126a84c51d
Jul 24 19:20:51.913604 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:51.913582    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"
Jul 24 19:20:52.050486 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:52.050462    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"]
Jul 24 19:20:52.052063 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:20:52.052038    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c34b3f_f5ff_4ed8_b7fd_fe64af63c395.slice/crio-03d5008e2aea15e3f0ff8d3a2211df5f01ef8c35a1a1083c244c3580732f8cf8 WatchSource:0}: Error finding container 03d5008e2aea15e3f0ff8d3a2211df5f01ef8c35a1a1083c244c3580732f8cf8: Status 404 returned error can't find the container with id 03d5008e2aea15e3f0ff8d3a2211df5f01ef8c35a1a1083c244c3580732f8cf8
Jul 24 19:20:52.721567 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:52.721527    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg" event={"ID":"bea8eb12-e592-4345-b6ef-e5e68de925cb","Type":"ContainerStarted","Data":"2d52ce7adb95b020d292c9ea4e86804a9bbe4a1b15fabb669034ec47fb80d20e"}
Jul 24 19:20:52.721783 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:52.721625    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"
Jul 24 19:20:52.721783 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:52.721648    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg" event={"ID":"bea8eb12-e592-4345-b6ef-e5e68de925cb","Type":"ContainerStarted","Data":"9188497cf0eb0fa03447533b5c5bd87fa3f0ebb35d3b1f7d5cea01126a84c51d"}
Jul 24 19:20:52.723370 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:52.723338    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b" event={"ID":"f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395","Type":"ContainerStarted","Data":"721fa532cdc2416e08689780488764e3a8fb8691dca2c86b12d2c8d64968c897"}
Jul 24 19:20:52.723520 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:52.723417    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b" event={"ID":"f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395","Type":"ContainerStarted","Data":"03d5008e2aea15e3f0ff8d3a2211df5f01ef8c35a1a1083c244c3580732f8cf8"}
Jul 24 19:20:52.723520 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:52.723449    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"
Jul 24 19:20:52.725118 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:52.725022    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn" event={"ID":"43533694-1e46-4364-a9d1-1ec037af912f","Type":"ContainerStarted","Data":"76da41786a9224e5f2d6bafc2bb2d76e5bc72fcc740389be44cf17dd91a91fe2"}
Jul 24 19:20:52.725118 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:52.725074    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn" event={"ID":"43533694-1e46-4364-a9d1-1ec037af912f","Type":"ContainerStarted","Data":"b424a7cdf464765d0dfba4b30680757ad52fd638fa14e2cf8b9bd95e2d72a5bf"}
Jul 24 19:20:52.725311 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:52.725158    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:20:52.738654 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:52.738606    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg" podStartSLOduration=1.738593019 podStartE2EDuration="1.738593019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:20:52.736796488 +0000 UTC m=+2191.127747966" watchObservedRunningTime="2026-07-24 19:20:52.738593019 +0000 UTC m=+2191.129544497"
Jul 24 19:20:52.755244 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:52.755189    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn" podStartSLOduration=2.345654092 podStartE2EDuration="2.755168481s" totalImagesPullingTime="409.514389ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:20:52.751976901 +0000 UTC m=+2191.142928378" watchObservedRunningTime="2026-07-24 19:20:52.755168481 +0000 UTC m=+2191.146119961"
Jul 24 19:20:52.766914 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:52.766865    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b" podStartSLOduration=1.766849844 podStartE2EDuration="1.766849844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:20:52.765463385 +0000 UTC m=+2191.156414865" watchObservedRunningTime="2026-07-24 19:20:52.766849844 +0000 UTC m=+2191.157801320"
Jul 24 19:20:56.358934 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:56.358901    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:20:58.738035 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:58.738009    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:20:59.078657 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.078622    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"]
Jul 24 19:20:59.082417 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.082379    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:20:59.084862 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.084840    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-ff6bd9-serving-cert\""
Jul 24 19:20:59.091887 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.091863    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"]
Jul 24 19:20:59.151250 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.151214    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8wns\" (UniqueName: \"kubernetes.io/projected/3d095b9b-2d37-4683-898d-8723f3921aa5-kube-api-access-z8wns\") pod \"maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr\" (UID: \"3d095b9b-2d37-4683-898d-8723f3921aa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:20:59.151476 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.151260    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d095b9b-2d37-4683-898d-8723f3921aa5-maas-api-tls\") pod \"maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr\" (UID: \"3d095b9b-2d37-4683-898d-8723f3921aa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:20:59.151476 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.151372    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d095b9b-2d37-4683-898d-8723f3921aa5-openshift-service-ca\") pod \"maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr\" (UID: \"3d095b9b-2d37-4683-898d-8723f3921aa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:20:59.251993 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.251960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d095b9b-2d37-4683-898d-8723f3921aa5-openshift-service-ca\") pod \"maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr\" (UID: \"3d095b9b-2d37-4683-898d-8723f3921aa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:20:59.252197 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.252009    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8wns\" (UniqueName: \"kubernetes.io/projected/3d095b9b-2d37-4683-898d-8723f3921aa5-kube-api-access-z8wns\") pod \"maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr\" (UID: \"3d095b9b-2d37-4683-898d-8723f3921aa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:20:59.252197 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.252149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d095b9b-2d37-4683-898d-8723f3921aa5-maas-api-tls\") pod \"maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr\" (UID: \"3d095b9b-2d37-4683-898d-8723f3921aa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:20:59.252330 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:20:59.252245    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-serving-cert: secret "maas-api-e2e-api-b-ff6bd9-serving-cert" not found
Jul 24 19:20:59.252330 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:20:59.252320    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d095b9b-2d37-4683-898d-8723f3921aa5-maas-api-tls podName:3d095b9b-2d37-4683-898d-8723f3921aa5 nodeName:}" failed. No retries permitted until 2026-07-24 19:20:59.752302659 +0000 UTC m=+2198.143254114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3d095b9b-2d37-4683-898d-8723f3921aa5-maas-api-tls") pod "maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr" (UID: "3d095b9b-2d37-4683-898d-8723f3921aa5") : secret "maas-api-e2e-api-b-ff6bd9-serving-cert" not found
Jul 24 19:20:59.252749 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.252725    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d095b9b-2d37-4683-898d-8723f3921aa5-openshift-service-ca\") pod \"maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr\" (UID: \"3d095b9b-2d37-4683-898d-8723f3921aa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:20:59.262494 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.262463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8wns\" (UniqueName: \"kubernetes.io/projected/3d095b9b-2d37-4683-898d-8723f3921aa5-kube-api-access-z8wns\") pod \"maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr\" (UID: \"3d095b9b-2d37-4683-898d-8723f3921aa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:20:59.756662 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.756624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d095b9b-2d37-4683-898d-8723f3921aa5-maas-api-tls\") pod \"maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr\" (UID: \"3d095b9b-2d37-4683-898d-8723f3921aa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:20:59.759339 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.759311    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d095b9b-2d37-4683-898d-8723f3921aa5-maas-api-tls\") pod \"maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr\" (UID: \"3d095b9b-2d37-4683-898d-8723f3921aa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:20:59.995534 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:20:59.995500    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:21:00.121332 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.117129    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"]
Jul 24 19:21:00.127679 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.127651    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"
Jul 24 19:21:00.132443 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.132418    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-ff6bd9-dockercfg-mcqt5\""
Jul 24 19:21:00.132917 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.132896    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-ff6bd9\""
Jul 24 19:21:00.157362 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.140776    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"]
Jul 24 19:21:00.162035 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.161813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e8ca30f-3b5e-4b67-8024-b4b5452d05f7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb\" (UID: \"3e8ca30f-3b5e-4b67-8024-b4b5452d05f7\") " pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"
Jul 24 19:21:00.162035 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.161889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-978pk\" (UniqueName: \"kubernetes.io/projected/3e8ca30f-3b5e-4b67-8024-b4b5452d05f7-kube-api-access-978pk\") pod \"payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb\" (UID: \"3e8ca30f-3b5e-4b67-8024-b4b5452d05f7\") " pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"
Jul 24 19:21:00.207826 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.207718    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"]
Jul 24 19:21:00.262965 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.262884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e8ca30f-3b5e-4b67-8024-b4b5452d05f7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb\" (UID: \"3e8ca30f-3b5e-4b67-8024-b4b5452d05f7\") " pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"
Jul 24 19:21:00.262965 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.262945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-978pk\" (UniqueName: \"kubernetes.io/projected/3e8ca30f-3b5e-4b67-8024-b4b5452d05f7-kube-api-access-978pk\") pod \"payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb\" (UID: \"3e8ca30f-3b5e-4b67-8024-b4b5452d05f7\") " pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"
Jul 24 19:21:00.263633 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.263609    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e8ca30f-3b5e-4b67-8024-b4b5452d05f7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb\" (UID: \"3e8ca30f-3b5e-4b67-8024-b4b5452d05f7\") " pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"
Jul 24 19:21:00.270928 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.270906    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-978pk\" (UniqueName: \"kubernetes.io/projected/3e8ca30f-3b5e-4b67-8024-b4b5452d05f7-kube-api-access-978pk\") pod \"payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb\" (UID: \"3e8ca30f-3b5e-4b67-8024-b4b5452d05f7\") " pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"
Jul 24 19:21:00.474549 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.474519    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"
Jul 24 19:21:00.652443 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.652410    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"]
Jul 24 19:21:00.655739 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:21:00.655706    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e8ca30f_3b5e_4b67_8024_b4b5452d05f7.slice/crio-ba1613f13c48cc31862560e14540c75b2c3209dee6314aa5468cf370af1e741f WatchSource:0}: Error finding container ba1613f13c48cc31862560e14540c75b2c3209dee6314aa5468cf370af1e741f: Status 404 returned error can't find the container with id ba1613f13c48cc31862560e14540c75b2c3209dee6314aa5468cf370af1e741f
Jul 24 19:21:00.769842 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.769814    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb" event={"ID":"3e8ca30f-3b5e-4b67-8024-b4b5452d05f7","Type":"ContainerStarted","Data":"ba1613f13c48cc31862560e14540c75b2c3209dee6314aa5468cf370af1e741f"}
Jul 24 19:21:00.771376 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.771351    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr" event={"ID":"3d095b9b-2d37-4683-898d-8723f3921aa5","Type":"ContainerStarted","Data":"588008f5aa77a7baf5586f46a5fe505587f2a7610353b2e2a321372966e2b671"}
Jul 24 19:21:00.771491 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.771434    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:21:00.789479 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:00.789431    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr" podStartSLOduration=1.364630589 podStartE2EDuration="1.789414613s" totalImagesPullingTime="424.784024ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:20:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:21:00.787638986 +0000 UTC m=+2199.178590476" watchObservedRunningTime="2026-07-24 19:21:00.789414613 +0000 UTC m=+2199.180366084"
Jul 24 19:21:01.780177 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:01.780145    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb" event={"ID":"3e8ca30f-3b5e-4b67-8024-b4b5452d05f7","Type":"ContainerStarted","Data":"c686e4422669c41e670a86e5a014eb0494b93532304a01460fea1aba52f571e8"}
Jul 24 19:21:01.780670 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:01.780252    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"
Jul 24 19:21:01.781862 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:01.781831    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr" event={"ID":"3d095b9b-2d37-4683-898d-8723f3921aa5","Type":"ContainerStarted","Data":"fc3a7470ba477e2fe50169d48e73e5e83b5837832b36ab9ec61b4eeaf3858b4e"}
Jul 24 19:21:01.796722 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:01.796676    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb" podStartSLOduration=1.796658605 podStartE2EDuration="1.796658605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:21:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:21:01.796420455 +0000 UTC m=+2200.187371934" watchObservedRunningTime="2026-07-24 19:21:01.796658605 +0000 UTC m=+2200.187610082"
Jul 24 19:21:03.732800 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:03.732769    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"
Jul 24 19:21:03.733206 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:03.732905    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"
Jul 24 19:21:04.703896 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:04.703808    2563 patch_prober.go:28] interesting pod/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 24 19:21:04.703896 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:04.703870    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8" podUID="99745325-561f-4a1d-afbe-bdb95274757f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 24 19:21:04.802511 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:04.802420    2563 generic.go:356] "Generic (PLEG): container finished" podID="99745325-561f-4a1d-afbe-bdb95274757f" containerID="f754b9a3c473e4717803f08458d9f33203627db02e6d8ea7a171f325d040f7f7" exitCode=0
Jul 24 19:21:04.802511 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:04.802462    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8" event={"ID":"99745325-561f-4a1d-afbe-bdb95274757f","Type":"ContainerDied","Data":"f754b9a3c473e4717803f08458d9f33203627db02e6d8ea7a171f325d040f7f7"}
Jul 24 19:21:04.802918 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:04.802828    2563 scope.go:117] "RemoveContainer" containerID="f754b9a3c473e4717803f08458d9f33203627db02e6d8ea7a171f325d040f7f7"
Jul 24 19:21:04.803956 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:04.803929    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:21:04.804017 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:04.804001    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:21:04.804053 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:04.804040    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:21:05.810655 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:05.810615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8" event={"ID":"99745325-561f-4a1d-afbe-bdb95274757f","Type":"ContainerStarted","Data":"a5da72a0966c05348dedc13f2c6db9ffec8a96a004c364ddf371268168983941"}
Jul 24 19:21:06.632528 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:06.632493    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:21:06.632935 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:06.632847    2563 patch_prober.go:28] interesting pod/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 24 19:21:06.633119 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:06.633094    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8" podUID="99745325-561f-4a1d-afbe-bdb95274757f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 24 19:21:07.633047 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:07.633005    2563 patch_prober.go:28] interesting pod/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 24 19:21:07.633576 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:07.633476    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8" podUID="99745325-561f-4a1d-afbe-bdb95274757f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 24 19:21:07.795966 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:07.795939    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:21:08.632628 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:08.632590    2563 patch_prober.go:28] interesting pod/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 24 19:21:08.632969 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:08.632941    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8" podUID="99745325-561f-4a1d-afbe-bdb95274757f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 24 19:21:09.632562 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:09.632523    2563 patch_prober.go:28] interesting pod/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 24 19:21:09.633157 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:09.633124    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8" podUID="99745325-561f-4a1d-afbe-bdb95274757f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 24 19:21:09.750738 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:09.750700    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:21:10.636826 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:10.636797    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:21:10.637289 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:10.637271    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:21:10.638080 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:10.638058    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:21:12.792264 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:12.792234    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"
Jul 24 19:21:26.727792 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:26.727744    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"]
Jul 24 19:21:26.728238 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:26.728027    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn" podUID="43533694-1e46-4364-a9d1-1ec037af912f" containerName="maas-api" containerID="cri-o://76da41786a9224e5f2d6bafc2bb2d76e5bc72fcc740389be44cf17dd91a91fe2" gracePeriod=30
Jul 24 19:21:26.788989 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:26.788956    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"]
Jul 24 19:21:26.789174 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:26.789151    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b" podUID="f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395" containerName="payload-processing" containerID="cri-o://721fa532cdc2416e08689780488764e3a8fb8691dca2c86b12d2c8d64968c897" gracePeriod=30
Jul 24 19:21:26.794558 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:26.794529    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"]
Jul 24 19:21:26.794804 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:26.794769    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg" podUID="bea8eb12-e592-4345-b6ef-e5e68de925cb" containerName="payload-pre-processing" containerID="cri-o://2d52ce7adb95b020d292c9ea4e86804a9bbe4a1b15fabb669034ec47fb80d20e" gracePeriod=30
Jul 24 19:21:26.930459 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:26.930422    2563 generic.go:356] "Generic (PLEG): container finished" podID="bea8eb12-e592-4345-b6ef-e5e68de925cb" containerID="2d52ce7adb95b020d292c9ea4e86804a9bbe4a1b15fabb669034ec47fb80d20e" exitCode=0
Jul 24 19:21:26.930695 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:26.930490    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg" event={"ID":"bea8eb12-e592-4345-b6ef-e5e68de925cb","Type":"ContainerDied","Data":"2d52ce7adb95b020d292c9ea4e86804a9bbe4a1b15fabb669034ec47fb80d20e"}
Jul 24 19:21:26.932430 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:26.932265    2563 generic.go:356] "Generic (PLEG): container finished" podID="f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395" containerID="721fa532cdc2416e08689780488764e3a8fb8691dca2c86b12d2c8d64968c897" exitCode=0
Jul 24 19:21:26.932430 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:26.932337    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b" event={"ID":"f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395","Type":"ContainerDied","Data":"721fa532cdc2416e08689780488764e3a8fb8691dca2c86b12d2c8d64968c897"}
Jul 24 19:21:26.934973 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:26.934439    2563 generic.go:356] "Generic (PLEG): container finished" podID="43533694-1e46-4364-a9d1-1ec037af912f" containerID="76da41786a9224e5f2d6bafc2bb2d76e5bc72fcc740389be44cf17dd91a91fe2" exitCode=0
Jul 24 19:21:26.934973 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:26.934499    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn" event={"ID":"43533694-1e46-4364-a9d1-1ec037af912f","Type":"ContainerDied","Data":"76da41786a9224e5f2d6bafc2bb2d76e5bc72fcc740389be44cf17dd91a91fe2"}
Jul 24 19:21:27.045387 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.045360    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:21:27.067689 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.067658    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"
Jul 24 19:21:27.076719 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.076695    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"
Jul 24 19:21:27.101108 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.101081    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43533694-1e46-4364-a9d1-1ec037af912f-openshift-service-ca\") pod \"43533694-1e46-4364-a9d1-1ec037af912f\" (UID: \"43533694-1e46-4364-a9d1-1ec037af912f\") "
Jul 24 19:21:27.101284 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.101134    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43533694-1e46-4364-a9d1-1ec037af912f-maas-api-tls\") pod \"43533694-1e46-4364-a9d1-1ec037af912f\" (UID: \"43533694-1e46-4364-a9d1-1ec037af912f\") "
Jul 24 19:21:27.101284 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.101190    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9vmd\" (UniqueName: \"kubernetes.io/projected/43533694-1e46-4364-a9d1-1ec037af912f-kube-api-access-j9vmd\") pod \"43533694-1e46-4364-a9d1-1ec037af912f\" (UID: \"43533694-1e46-4364-a9d1-1ec037af912f\") "
Jul 24 19:21:27.101872 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.101837    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43533694-1e46-4364-a9d1-1ec037af912f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "43533694-1e46-4364-a9d1-1ec037af912f" (UID: "43533694-1e46-4364-a9d1-1ec037af912f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:21:27.104193 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.104167    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43533694-1e46-4364-a9d1-1ec037af912f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "43533694-1e46-4364-a9d1-1ec037af912f" (UID: "43533694-1e46-4364-a9d1-1ec037af912f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:21:27.104606 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.104573    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43533694-1e46-4364-a9d1-1ec037af912f-kube-api-access-j9vmd" (OuterVolumeSpecName: "kube-api-access-j9vmd") pod "43533694-1e46-4364-a9d1-1ec037af912f" (UID: "43533694-1e46-4364-a9d1-1ec037af912f"). InnerVolumeSpecName "kube-api-access-j9vmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:21:27.201873 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.201836    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzdd8\" (UniqueName: \"kubernetes.io/projected/f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395-kube-api-access-pzdd8\") pod \"f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395\" (UID: \"f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395\") "
Jul 24 19:21:27.202068 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.201892    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395-plugins-config-volume\") pod \"f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395\" (UID: \"f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395\") "
Jul 24 19:21:27.202068 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.201939    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bea8eb12-e592-4345-b6ef-e5e68de925cb-plugins-config-volume\") pod \"bea8eb12-e592-4345-b6ef-e5e68de925cb\" (UID: \"bea8eb12-e592-4345-b6ef-e5e68de925cb\") "
Jul 24 19:21:27.202068 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.201973    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5d2fz\" (UniqueName: \"kubernetes.io/projected/bea8eb12-e592-4345-b6ef-e5e68de925cb-kube-api-access-5d2fz\") pod \"bea8eb12-e592-4345-b6ef-e5e68de925cb\" (UID: \"bea8eb12-e592-4345-b6ef-e5e68de925cb\") "
Jul 24 19:21:27.202227 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.202210    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43533694-1e46-4364-a9d1-1ec037af912f-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:27.202270 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.202234    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43533694-1e46-4364-a9d1-1ec037af912f-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:27.202270 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.202249    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9vmd\" (UniqueName: \"kubernetes.io/projected/43533694-1e46-4364-a9d1-1ec037af912f-kube-api-access-j9vmd\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:27.202373 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.202347    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bea8eb12-e592-4345-b6ef-e5e68de925cb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bea8eb12-e592-4345-b6ef-e5e68de925cb" (UID: "bea8eb12-e592-4345-b6ef-e5e68de925cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:21:27.202463 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.202347    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395" (UID: "f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:21:27.204209 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.204187    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395-kube-api-access-pzdd8" (OuterVolumeSpecName: "kube-api-access-pzdd8") pod "f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395" (UID: "f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395"). InnerVolumeSpecName "kube-api-access-pzdd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:21:27.204267 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.204230    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bea8eb12-e592-4345-b6ef-e5e68de925cb-kube-api-access-5d2fz" (OuterVolumeSpecName: "kube-api-access-5d2fz") pod "bea8eb12-e592-4345-b6ef-e5e68de925cb" (UID: "bea8eb12-e592-4345-b6ef-e5e68de925cb"). InnerVolumeSpecName "kube-api-access-5d2fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:21:27.303532 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.303446    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:27.303532 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.303475    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bea8eb12-e592-4345-b6ef-e5e68de925cb-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:27.303532 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.303485    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5d2fz\" (UniqueName: \"kubernetes.io/projected/bea8eb12-e592-4345-b6ef-e5e68de925cb-kube-api-access-5d2fz\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:27.303532 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.303494    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzdd8\" (UniqueName: \"kubernetes.io/projected/f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395-kube-api-access-pzdd8\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:27.941439 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.941404    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"
Jul 24 19:21:27.941913 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.941435    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg" event={"ID":"bea8eb12-e592-4345-b6ef-e5e68de925cb","Type":"ContainerDied","Data":"9188497cf0eb0fa03447533b5c5bd87fa3f0ebb35d3b1f7d5cea01126a84c51d"}
Jul 24 19:21:27.941913 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.941505    2563 scope.go:117] "RemoveContainer" containerID="2d52ce7adb95b020d292c9ea4e86804a9bbe4a1b15fabb669034ec47fb80d20e"
Jul 24 19:21:27.942980 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.942946    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"
Jul 24 19:21:27.943061 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.942969    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b" event={"ID":"f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395","Type":"ContainerDied","Data":"03d5008e2aea15e3f0ff8d3a2211df5f01ef8c35a1a1083c244c3580732f8cf8"}
Jul 24 19:21:27.944525 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.944501    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn" event={"ID":"43533694-1e46-4364-a9d1-1ec037af912f","Type":"ContainerDied","Data":"b424a7cdf464765d0dfba4b30680757ad52fd638fa14e2cf8b9bd95e2d72a5bf"}
Jul 24 19:21:27.944645 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.944554    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"
Jul 24 19:21:27.953265 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.953232    2563 scope.go:117] "RemoveContainer" containerID="721fa532cdc2416e08689780488764e3a8fb8691dca2c86b12d2c8d64968c897"
Jul 24 19:21:27.965854 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.965832    2563 scope.go:117] "RemoveContainer" containerID="76da41786a9224e5f2d6bafc2bb2d76e5bc72fcc740389be44cf17dd91a91fe2"
Jul 24 19:21:27.973023 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.973001    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"]
Jul 24 19:21:27.977580 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.977561    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4bd2a2-7d785f59d8-wxwrg"]
Jul 24 19:21:27.981902 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:21:27.981873    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c34b3f_f5ff_4ed8_b7fd_fe64af63c395.slice/crio-03d5008e2aea15e3f0ff8d3a2211df5f01ef8c35a1a1083c244c3580732f8cf8\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbea8eb12_e592_4345_b6ef_e5e68de925cb.slice/crio-9188497cf0eb0fa03447533b5c5bd87fa3f0ebb35d3b1f7d5cea01126a84c51d\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43533694_1e46_4364_a9d1_1ec037af912f.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c34b3f_f5ff_4ed8_b7fd_fe64af63c395.slice\": RecentStats: unable to find data in memory cache]"
Jul 24 19:21:27.989614 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.989588    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"]
Jul 24 19:21:27.994093 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:27.994030    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4bd2a2-5f6589bb7b-8pmnn"]
Jul 24 19:21:28.005964 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:28.005943    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"]
Jul 24 19:21:28.010505 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:28.010487    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4bd2a2-dc9b999f8-4qf2b"]
Jul 24 19:21:28.204077 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:28.204043    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43533694-1e46-4364-a9d1-1ec037af912f" path="/var/lib/kubelet/pods/43533694-1e46-4364-a9d1-1ec037af912f/volumes"
Jul 24 19:21:28.204436 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:28.204423    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bea8eb12-e592-4345-b6ef-e5e68de925cb" path="/var/lib/kubelet/pods/bea8eb12-e592-4345-b6ef-e5e68de925cb/volumes"
Jul 24 19:21:28.204723 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:28.204712    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395" path="/var/lib/kubelet/pods/f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395/volumes"
Jul 24 19:21:33.552216 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:33.552174    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:21:37.895893 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:37.895862    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"]
Jul 24 19:21:37.896310 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:37.896135    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8" podUID="99745325-561f-4a1d-afbe-bdb95274757f" containerName="istio-proxy" containerID="cri-o://a5da72a0966c05348dedc13f2c6db9ffec8a96a004c364ddf371268168983941" gracePeriod=30
Jul 24 19:21:40.639299 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:40.639256    2563 patch_prober.go:28] interesting pod/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 19:21:40.639727 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:40.639327    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8" podUID="99745325-561f-4a1d-afbe-bdb95274757f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 19:21:43.031095 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.031062    2563 generic.go:356] "Generic (PLEG): container finished" podID="99745325-561f-4a1d-afbe-bdb95274757f" containerID="a5da72a0966c05348dedc13f2c6db9ffec8a96a004c364ddf371268168983941" exitCode=0
Jul 24 19:21:43.031614 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.031135    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8" event={"ID":"99745325-561f-4a1d-afbe-bdb95274757f","Type":"ContainerDied","Data":"a5da72a0966c05348dedc13f2c6db9ffec8a96a004c364ddf371268168983941"}
Jul 24 19:21:43.031614 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.031202    2563 scope.go:117] "RemoveContainer" containerID="f754b9a3c473e4717803f08458d9f33203627db02e6d8ea7a171f325d040f7f7"
Jul 24 19:21:43.144904 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.144830    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:21:43.242759 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.242654    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-workload-certs\") pod \"99745325-561f-4a1d-afbe-bdb95274757f\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") "
Jul 24 19:21:43.242759 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.242712    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-credential-socket\") pod \"99745325-561f-4a1d-afbe-bdb95274757f\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") "
Jul 24 19:21:43.242759 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.242735    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99745325-561f-4a1d-afbe-bdb95274757f-istio-token\") pod \"99745325-561f-4a1d-afbe-bdb95274757f\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") "
Jul 24 19:21:43.243050 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.242767    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99745325-561f-4a1d-afbe-bdb95274757f-istio-podinfo\") pod \"99745325-561f-4a1d-afbe-bdb95274757f\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") "
Jul 24 19:21:43.243050 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.242817    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-257pv\" (UniqueName: \"kubernetes.io/projected/99745325-561f-4a1d-afbe-bdb95274757f-kube-api-access-257pv\") pod \"99745325-561f-4a1d-afbe-bdb95274757f\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") "
Jul 24 19:21:43.243050 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.242856    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-istio-data\") pod \"99745325-561f-4a1d-afbe-bdb95274757f\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") "
Jul 24 19:21:43.243050 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.242882    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99745325-561f-4a1d-afbe-bdb95274757f-istiod-ca-cert\") pod \"99745325-561f-4a1d-afbe-bdb95274757f\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") "
Jul 24 19:21:43.243050 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.242911    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-workload-socket\") pod \"99745325-561f-4a1d-afbe-bdb95274757f\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") "
Jul 24 19:21:43.243050 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.242941    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-istio-envoy\") pod \"99745325-561f-4a1d-afbe-bdb95274757f\" (UID: \"99745325-561f-4a1d-afbe-bdb95274757f\") "
Jul 24 19:21:43.243050 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.242980    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "99745325-561f-4a1d-afbe-bdb95274757f" (UID: "99745325-561f-4a1d-afbe-bdb95274757f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:21:43.243447 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.243109    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "99745325-561f-4a1d-afbe-bdb95274757f" (UID: "99745325-561f-4a1d-afbe-bdb95274757f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:21:43.243447 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.243230    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-workload-certs\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:43.243447 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.243247    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-credential-socket\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:43.243447 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.243358    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "99745325-561f-4a1d-afbe-bdb95274757f" (UID: "99745325-561f-4a1d-afbe-bdb95274757f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:21:43.243830 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.243543    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99745325-561f-4a1d-afbe-bdb95274757f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "99745325-561f-4a1d-afbe-bdb95274757f" (UID: "99745325-561f-4a1d-afbe-bdb95274757f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:21:43.243830 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.243669    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-istio-data" (OuterVolumeSpecName: "istio-data") pod "99745325-561f-4a1d-afbe-bdb95274757f" (UID: "99745325-561f-4a1d-afbe-bdb95274757f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:21:43.245203 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.245177    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99745325-561f-4a1d-afbe-bdb95274757f-istio-token" (OuterVolumeSpecName: "istio-token") pod "99745325-561f-4a1d-afbe-bdb95274757f" (UID: "99745325-561f-4a1d-afbe-bdb95274757f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:21:43.245310 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.245227    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99745325-561f-4a1d-afbe-bdb95274757f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "99745325-561f-4a1d-afbe-bdb95274757f" (UID: "99745325-561f-4a1d-afbe-bdb95274757f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 19:21:43.245310 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.245282    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99745325-561f-4a1d-afbe-bdb95274757f-kube-api-access-257pv" (OuterVolumeSpecName: "kube-api-access-257pv") pod "99745325-561f-4a1d-afbe-bdb95274757f" (UID: "99745325-561f-4a1d-afbe-bdb95274757f"). InnerVolumeSpecName "kube-api-access-257pv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:21:43.245310 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.245284    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "99745325-561f-4a1d-afbe-bdb95274757f" (UID: "99745325-561f-4a1d-afbe-bdb95274757f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:21:43.343778 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.343738    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-istio-data\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:43.343778 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.343772    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99745325-561f-4a1d-afbe-bdb95274757f-istiod-ca-cert\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:43.343778 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.343781    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-workload-socket\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:43.343778 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.343790    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99745325-561f-4a1d-afbe-bdb95274757f-istio-envoy\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:43.344053 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.343799    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99745325-561f-4a1d-afbe-bdb95274757f-istio-token\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:43.344053 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.343807    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99745325-561f-4a1d-afbe-bdb95274757f-istio-podinfo\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:43.344053 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:43.343815    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-257pv\" (UniqueName: \"kubernetes.io/projected/99745325-561f-4a1d-afbe-bdb95274757f-kube-api-access-257pv\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:44.037666 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:44.037626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8" event={"ID":"99745325-561f-4a1d-afbe-bdb95274757f","Type":"ContainerDied","Data":"456bcfc10a21fe658a874ed71abb7f019b8af1327b0993354d417f8075097bb6"}
Jul 24 19:21:44.037666 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:44.037648    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"
Jul 24 19:21:44.038164 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:44.037696    2563 scope.go:117] "RemoveContainer" containerID="a5da72a0966c05348dedc13f2c6db9ffec8a96a004c364ddf371268168983941"
Jul 24 19:21:44.064519 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:44.064488    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"]
Jul 24 19:21:44.075607 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:44.075578    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-4bd2a2-openshift-default-7b8b768886-tzzz8"]
Jul 24 19:21:44.208213 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:44.208174    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99745325-561f-4a1d-afbe-bdb95274757f" path="/var/lib/kubelet/pods/99745325-561f-4a1d-afbe-bdb95274757f/volumes"
Jul 24 19:21:53.713927 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:53.713887    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"]
Jul 24 19:21:53.714431 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:53.714153    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr" podUID="3d095b9b-2d37-4683-898d-8723f3921aa5" containerName="maas-api" containerID="cri-o://fc3a7470ba477e2fe50169d48e73e5e83b5837832b36ab9ec61b4eeaf3858b4e" gracePeriod=30
Jul 24 19:21:53.769440 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:53.769379    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"]
Jul 24 19:21:53.769731 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:53.769674    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb" podUID="3e8ca30f-3b5e-4b67-8024-b4b5452d05f7" containerName="payload-processing" containerID="cri-o://c686e4422669c41e670a86e5a014eb0494b93532304a01460fea1aba52f571e8" gracePeriod=30
Jul 24 19:21:54.014740 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.014716    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:21:54.031680 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.031462    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d095b9b-2d37-4683-898d-8723f3921aa5-maas-api-tls\") pod \"3d095b9b-2d37-4683-898d-8723f3921aa5\" (UID: \"3d095b9b-2d37-4683-898d-8723f3921aa5\") "
Jul 24 19:21:54.031680 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.031517    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8wns\" (UniqueName: \"kubernetes.io/projected/3d095b9b-2d37-4683-898d-8723f3921aa5-kube-api-access-z8wns\") pod \"3d095b9b-2d37-4683-898d-8723f3921aa5\" (UID: \"3d095b9b-2d37-4683-898d-8723f3921aa5\") "
Jul 24 19:21:54.033839 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.031588    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d095b9b-2d37-4683-898d-8723f3921aa5-openshift-service-ca\") pod \"3d095b9b-2d37-4683-898d-8723f3921aa5\" (UID: \"3d095b9b-2d37-4683-898d-8723f3921aa5\") "
Jul 24 19:21:54.033995 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.033609    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d095b9b-2d37-4683-898d-8723f3921aa5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3d095b9b-2d37-4683-898d-8723f3921aa5" (UID: "3d095b9b-2d37-4683-898d-8723f3921aa5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:21:54.034427 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.034299    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d095b9b-2d37-4683-898d-8723f3921aa5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3d095b9b-2d37-4683-898d-8723f3921aa5" (UID: "3d095b9b-2d37-4683-898d-8723f3921aa5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:21:54.036506 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.036302    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d095b9b-2d37-4683-898d-8723f3921aa5-kube-api-access-z8wns" (OuterVolumeSpecName: "kube-api-access-z8wns") pod "3d095b9b-2d37-4683-898d-8723f3921aa5" (UID: "3d095b9b-2d37-4683-898d-8723f3921aa5"). InnerVolumeSpecName "kube-api-access-z8wns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:21:54.045348 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.045329    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"
Jul 24 19:21:54.087742 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.087712    2563 generic.go:356] "Generic (PLEG): container finished" podID="3e8ca30f-3b5e-4b67-8024-b4b5452d05f7" containerID="c686e4422669c41e670a86e5a014eb0494b93532304a01460fea1aba52f571e8" exitCode=0
Jul 24 19:21:54.087890 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.087753    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"
Jul 24 19:21:54.087890 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.087778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb" event={"ID":"3e8ca30f-3b5e-4b67-8024-b4b5452d05f7","Type":"ContainerDied","Data":"c686e4422669c41e670a86e5a014eb0494b93532304a01460fea1aba52f571e8"}
Jul 24 19:21:54.087890 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.087827    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb" event={"ID":"3e8ca30f-3b5e-4b67-8024-b4b5452d05f7","Type":"ContainerDied","Data":"ba1613f13c48cc31862560e14540c75b2c3209dee6314aa5468cf370af1e741f"}
Jul 24 19:21:54.087890 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.087856    2563 scope.go:117] "RemoveContainer" containerID="c686e4422669c41e670a86e5a014eb0494b93532304a01460fea1aba52f571e8"
Jul 24 19:21:54.089092 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.089070    2563 generic.go:356] "Generic (PLEG): container finished" podID="3d095b9b-2d37-4683-898d-8723f3921aa5" containerID="fc3a7470ba477e2fe50169d48e73e5e83b5837832b36ab9ec61b4eeaf3858b4e" exitCode=0
Jul 24 19:21:54.089182 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.089099    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr" event={"ID":"3d095b9b-2d37-4683-898d-8723f3921aa5","Type":"ContainerDied","Data":"fc3a7470ba477e2fe50169d48e73e5e83b5837832b36ab9ec61b4eeaf3858b4e"}
Jul 24 19:21:54.089182 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.089139    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"
Jul 24 19:21:54.089267 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.089145    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr" event={"ID":"3d095b9b-2d37-4683-898d-8723f3921aa5","Type":"ContainerDied","Data":"588008f5aa77a7baf5586f46a5fe505587f2a7610353b2e2a321372966e2b671"}
Jul 24 19:21:54.097301 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.097284    2563 scope.go:117] "RemoveContainer" containerID="c686e4422669c41e670a86e5a014eb0494b93532304a01460fea1aba52f571e8"
Jul 24 19:21:54.097576 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:21:54.097556    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c686e4422669c41e670a86e5a014eb0494b93532304a01460fea1aba52f571e8\": container with ID starting with c686e4422669c41e670a86e5a014eb0494b93532304a01460fea1aba52f571e8 not found: ID does not exist" containerID="c686e4422669c41e670a86e5a014eb0494b93532304a01460fea1aba52f571e8"
Jul 24 19:21:54.097675 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.097582    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c686e4422669c41e670a86e5a014eb0494b93532304a01460fea1aba52f571e8"} err="failed to get container status \"c686e4422669c41e670a86e5a014eb0494b93532304a01460fea1aba52f571e8\": rpc error: code = NotFound desc = could not find container \"c686e4422669c41e670a86e5a014eb0494b93532304a01460fea1aba52f571e8\": container with ID starting with c686e4422669c41e670a86e5a014eb0494b93532304a01460fea1aba52f571e8 not found: ID does not exist"
Jul 24 19:21:54.097675 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.097598    2563 scope.go:117] "RemoveContainer" containerID="fc3a7470ba477e2fe50169d48e73e5e83b5837832b36ab9ec61b4eeaf3858b4e"
Jul 24 19:21:54.105306 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.105289    2563 scope.go:117] "RemoveContainer" containerID="fc3a7470ba477e2fe50169d48e73e5e83b5837832b36ab9ec61b4eeaf3858b4e"
Jul 24 19:21:54.105563 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:21:54.105545    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc3a7470ba477e2fe50169d48e73e5e83b5837832b36ab9ec61b4eeaf3858b4e\": container with ID starting with fc3a7470ba477e2fe50169d48e73e5e83b5837832b36ab9ec61b4eeaf3858b4e not found: ID does not exist" containerID="fc3a7470ba477e2fe50169d48e73e5e83b5837832b36ab9ec61b4eeaf3858b4e"
Jul 24 19:21:54.105638 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.105569    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc3a7470ba477e2fe50169d48e73e5e83b5837832b36ab9ec61b4eeaf3858b4e"} err="failed to get container status \"fc3a7470ba477e2fe50169d48e73e5e83b5837832b36ab9ec61b4eeaf3858b4e\": rpc error: code = NotFound desc = could not find container \"fc3a7470ba477e2fe50169d48e73e5e83b5837832b36ab9ec61b4eeaf3858b4e\": container with ID starting with fc3a7470ba477e2fe50169d48e73e5e83b5837832b36ab9ec61b4eeaf3858b4e not found: ID does not exist"
Jul 24 19:21:54.113913 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.113893    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"]
Jul 24 19:21:54.118271 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.118245    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ff6bd9-75db9d4f55-dgrcr"]
Jul 24 19:21:54.134625 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.134603    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e8ca30f-3b5e-4b67-8024-b4b5452d05f7-plugins-config-volume\") pod \"3e8ca30f-3b5e-4b67-8024-b4b5452d05f7\" (UID: \"3e8ca30f-3b5e-4b67-8024-b4b5452d05f7\") "
Jul 24 19:21:54.134735 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.134654    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-978pk\" (UniqueName: \"kubernetes.io/projected/3e8ca30f-3b5e-4b67-8024-b4b5452d05f7-kube-api-access-978pk\") pod \"3e8ca30f-3b5e-4b67-8024-b4b5452d05f7\" (UID: \"3e8ca30f-3b5e-4b67-8024-b4b5452d05f7\") "
Jul 24 19:21:54.134874 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.134858    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d095b9b-2d37-4683-898d-8723f3921aa5-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:54.134941 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.134878    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d095b9b-2d37-4683-898d-8723f3921aa5-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:54.134941 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.134895    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8wns\" (UniqueName: \"kubernetes.io/projected/3d095b9b-2d37-4683-898d-8723f3921aa5-kube-api-access-z8wns\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:54.135013 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.134948    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e8ca30f-3b5e-4b67-8024-b4b5452d05f7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e8ca30f-3b5e-4b67-8024-b4b5452d05f7" (UID: "3e8ca30f-3b5e-4b67-8024-b4b5452d05f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:21:54.136782 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.136762    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e8ca30f-3b5e-4b67-8024-b4b5452d05f7-kube-api-access-978pk" (OuterVolumeSpecName: "kube-api-access-978pk") pod "3e8ca30f-3b5e-4b67-8024-b4b5452d05f7" (UID: "3e8ca30f-3b5e-4b67-8024-b4b5452d05f7"). InnerVolumeSpecName "kube-api-access-978pk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:21:54.203841 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.203805    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d095b9b-2d37-4683-898d-8723f3921aa5" path="/var/lib/kubelet/pods/3d095b9b-2d37-4683-898d-8723f3921aa5/volumes"
Jul 24 19:21:54.235810 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.235746    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e8ca30f-3b5e-4b67-8024-b4b5452d05f7-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:54.235810 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.235772    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-978pk\" (UniqueName: \"kubernetes.io/projected/3e8ca30f-3b5e-4b67-8024-b4b5452d05f7-kube-api-access-978pk\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:21:54.403755 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.403722    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"]
Jul 24 19:21:54.407788 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:54.407760    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ff6bd9-6df8ccd844-bq5pb"]
Jul 24 19:21:56.203493 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:56.203456    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e8ca30f-3b5e-4b67-8024-b4b5452d05f7" path="/var/lib/kubelet/pods/3e8ca30f-3b5e-4b67-8024-b4b5452d05f7/volumes"
Jul 24 19:21:58.152584 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:21:58.152539    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:22:05.667978 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.667941    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"]
Jul 24 19:22:05.668351 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668277    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395" containerName="payload-processing"
Jul 24 19:22:05.668351 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668290    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395" containerName="payload-processing"
Jul 24 19:22:05.668351 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668298    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99745325-561f-4a1d-afbe-bdb95274757f" containerName="istio-proxy"
Jul 24 19:22:05.668351 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668306    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="99745325-561f-4a1d-afbe-bdb95274757f" containerName="istio-proxy"
Jul 24 19:22:05.668351 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668320    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bea8eb12-e592-4345-b6ef-e5e68de925cb" containerName="payload-pre-processing"
Jul 24 19:22:05.668351 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668326    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bea8eb12-e592-4345-b6ef-e5e68de925cb" containerName="payload-pre-processing"
Jul 24 19:22:05.668351 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668335    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d095b9b-2d37-4683-898d-8723f3921aa5" containerName="maas-api"
Jul 24 19:22:05.668351 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668340    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d095b9b-2d37-4683-898d-8723f3921aa5" containerName="maas-api"
Jul 24 19:22:05.668351 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668347    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43533694-1e46-4364-a9d1-1ec037af912f" containerName="maas-api"
Jul 24 19:22:05.668351 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668352    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="43533694-1e46-4364-a9d1-1ec037af912f" containerName="maas-api"
Jul 24 19:22:05.668712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668358    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e8ca30f-3b5e-4b67-8024-b4b5452d05f7" containerName="payload-processing"
Jul 24 19:22:05.668712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668363    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e8ca30f-3b5e-4b67-8024-b4b5452d05f7" containerName="payload-processing"
Jul 24 19:22:05.668712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668423    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="43533694-1e46-4364-a9d1-1ec037af912f" containerName="maas-api"
Jul 24 19:22:05.668712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668433    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8c34b3f-f5ff-4ed8-b7fd-fe64af63c395" containerName="payload-processing"
Jul 24 19:22:05.668712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668440    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="99745325-561f-4a1d-afbe-bdb95274757f" containerName="istio-proxy"
Jul 24 19:22:05.668712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668446    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="99745325-561f-4a1d-afbe-bdb95274757f" containerName="istio-proxy"
Jul 24 19:22:05.668712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668451    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d095b9b-2d37-4683-898d-8723f3921aa5" containerName="maas-api"
Jul 24 19:22:05.668712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668458    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e8ca30f-3b5e-4b67-8024-b4b5452d05f7" containerName="payload-processing"
Jul 24 19:22:05.668712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.668467    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bea8eb12-e592-4345-b6ef-e5e68de925cb" containerName="payload-pre-processing"
Jul 24 19:22:05.671527 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.671509    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.674729 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.674703    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-e05f4d-openshift-default-dockercfg-l828b\""
Jul 24 19:22:05.697762 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.697730    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"]
Jul 24 19:22:05.726848 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.726811    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-workload-socket\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.726848 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.726849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-istio-data\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.727135 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.726880    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-credential-socket\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.727135 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.726912    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91816a65-55c7-452d-ba71-26266c4908d4-istio-podinfo\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.727135 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.726990    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-workload-certs\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.727135 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.727028    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91816a65-55c7-452d-ba71-26266c4908d4-istio-token\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.727135 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.727104    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njmp9\" (UniqueName: \"kubernetes.io/projected/91816a65-55c7-452d-ba71-26266c4908d4-kube-api-access-njmp9\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.727435 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.727162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91816a65-55c7-452d-ba71-26266c4908d4-istiod-ca-cert\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.727435 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.727196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-istio-envoy\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.828671 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.828632    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-workload-socket\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.828671 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.828669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-istio-data\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.828926 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.828689    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-credential-socket\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.828926 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.828711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91816a65-55c7-452d-ba71-26266c4908d4-istio-podinfo\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.828926 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.828728    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-workload-certs\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.828926 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.828744    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91816a65-55c7-452d-ba71-26266c4908d4-istio-token\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.828926 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.828793    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njmp9\" (UniqueName: \"kubernetes.io/projected/91816a65-55c7-452d-ba71-26266c4908d4-kube-api-access-njmp9\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.828926 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.828838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91816a65-55c7-452d-ba71-26266c4908d4-istiod-ca-cert\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.829250 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.829107    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-credential-socket\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.829250 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.829116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-istio-envoy\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.829250 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.829153    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-istio-data\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.829250 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.829171    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-workload-socket\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.829463 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.829258    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-workload-certs\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.829622 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.829603    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91816a65-55c7-452d-ba71-26266c4908d4-istiod-ca-cert\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.831337 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.831306    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-istio-envoy\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.831542 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.831519    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91816a65-55c7-452d-ba71-26266c4908d4-istio-podinfo\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.837067 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.837041    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91816a65-55c7-452d-ba71-26266c4908d4-istio-token\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.837671 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.837650    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njmp9\" (UniqueName: \"kubernetes.io/projected/91816a65-55c7-452d-ba71-26266c4908d4-kube-api-access-njmp9\") pod \"e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") " pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:05.981613 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:05.981511    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:06.142840 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:06.142817    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"]
Jul 24 19:22:06.144898 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:22:06.144864    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91816a65_55c7_452d_ba71_26266c4908d4.slice/crio-04ddc363afa14cac1e2572211509712e77f6ffcc823dec9d2777f0e9eb42fd96 WatchSource:0}: Error finding container 04ddc363afa14cac1e2572211509712e77f6ffcc823dec9d2777f0e9eb42fd96: Status 404 returned error can't find the container with id 04ddc363afa14cac1e2572211509712e77f6ffcc823dec9d2777f0e9eb42fd96
Jul 24 19:22:06.147075 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:06.147042    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:22:06.147174 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:06.147109    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:22:06.147174 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:06.147137    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:22:06.153039 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:06.153019    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw" event={"ID":"91816a65-55c7-452d-ba71-26266c4908d4","Type":"ContainerStarted","Data":"04ddc363afa14cac1e2572211509712e77f6ffcc823dec9d2777f0e9eb42fd96"}
Jul 24 19:22:07.159146 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:07.159113    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw" event={"ID":"91816a65-55c7-452d-ba71-26266c4908d4","Type":"ContainerStarted","Data":"9795d2cc7a7950150689e90e8d3a3e2bb895bce91a172f6084e7a76c04f5c903"}
Jul 24 19:22:07.196918 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:07.196868    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw" podStartSLOduration=2.196853771 podStartE2EDuration="2.196853771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:22:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:22:07.196666505 +0000 UTC m=+2265.587618018" watchObservedRunningTime="2026-07-24 19:22:07.196853771 +0000 UTC m=+2265.587805248"
Jul 24 19:22:07.982223 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:07.982182    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:07.986961 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:07.986933    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:08.167759 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:08.167721    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:08.168856 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:08.168836    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:09.199497 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.199468    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"]
Jul 24 19:22:09.199876 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.199856    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99745325-561f-4a1d-afbe-bdb95274757f" containerName="istio-proxy"
Jul 24 19:22:09.199876 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.199871    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="99745325-561f-4a1d-afbe-bdb95274757f" containerName="istio-proxy"
Jul 24 19:22:09.202860 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.202837    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:22:09.205174 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.205155    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-e05f4d-serving-cert\""
Jul 24 19:22:09.205174 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.205169    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 19:22:09.205460 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.205445    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 19:22:09.206137 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.206119    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cvjqn\""
Jul 24 19:22:09.214344 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.214321    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"]
Jul 24 19:22:09.260929 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.260900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/842bf3a4-5e60-4045-a932-6f569740ee01-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv\" (UID: \"842bf3a4-5e60-4045-a932-6f569740ee01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:22:09.261066 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.260953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtbdm\" (UniqueName: \"kubernetes.io/projected/842bf3a4-5e60-4045-a932-6f569740ee01-kube-api-access-qtbdm\") pod \"maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv\" (UID: \"842bf3a4-5e60-4045-a932-6f569740ee01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:22:09.261066 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.260989    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/842bf3a4-5e60-4045-a932-6f569740ee01-maas-api-tls\") pod \"maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv\" (UID: \"842bf3a4-5e60-4045-a932-6f569740ee01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:22:09.362324 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.362289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/842bf3a4-5e60-4045-a932-6f569740ee01-maas-api-tls\") pod \"maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv\" (UID: \"842bf3a4-5e60-4045-a932-6f569740ee01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:22:09.362533 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.362367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/842bf3a4-5e60-4045-a932-6f569740ee01-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv\" (UID: \"842bf3a4-5e60-4045-a932-6f569740ee01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:22:09.362533 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.362419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtbdm\" (UniqueName: \"kubernetes.io/projected/842bf3a4-5e60-4045-a932-6f569740ee01-kube-api-access-qtbdm\") pod \"maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv\" (UID: \"842bf3a4-5e60-4045-a932-6f569740ee01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:22:09.362533 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:22:09.362463    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-serving-cert: secret "maas-api-e2e-shared-a-e05f4d-serving-cert" not found
Jul 24 19:22:09.362663 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:22:09.362539    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/842bf3a4-5e60-4045-a932-6f569740ee01-maas-api-tls podName:842bf3a4-5e60-4045-a932-6f569740ee01 nodeName:}" failed. No retries permitted until 2026-07-24 19:22:09.862518656 +0000 UTC m=+2268.253470131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/842bf3a4-5e60-4045-a932-6f569740ee01-maas-api-tls") pod "maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv" (UID: "842bf3a4-5e60-4045-a932-6f569740ee01") : secret "maas-api-e2e-shared-a-e05f4d-serving-cert" not found
Jul 24 19:22:09.363058 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.363041    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/842bf3a4-5e60-4045-a932-6f569740ee01-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv\" (UID: \"842bf3a4-5e60-4045-a932-6f569740ee01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:22:09.371570 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.371546    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtbdm\" (UniqueName: \"kubernetes.io/projected/842bf3a4-5e60-4045-a932-6f569740ee01-kube-api-access-qtbdm\") pod \"maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv\" (UID: \"842bf3a4-5e60-4045-a932-6f569740ee01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:22:09.867779 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.867747    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/842bf3a4-5e60-4045-a932-6f569740ee01-maas-api-tls\") pod \"maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv\" (UID: \"842bf3a4-5e60-4045-a932-6f569740ee01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:22:09.870538 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.870517    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/842bf3a4-5e60-4045-a932-6f569740ee01-maas-api-tls\") pod \"maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv\" (UID: \"842bf3a4-5e60-4045-a932-6f569740ee01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:22:09.962633 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.962601    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"]
Jul 24 19:22:09.965956 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.965939    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"
Jul 24 19:22:09.968423 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.968388    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-e05f4d-dockercfg-rhn57\""
Jul 24 19:22:09.968548 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.968405    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-e05f4d\""
Jul 24 19:22:09.977951 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:09.977931    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"]
Jul 24 19:22:10.069739 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:10.069703    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78a8e8a6-da1d-4685-b288-0092732f0458-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk\" (UID: \"78a8e8a6-da1d-4685-b288-0092732f0458\") " pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"
Jul 24 19:22:10.069739 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:10.069738    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdpzv\" (UniqueName: \"kubernetes.io/projected/78a8e8a6-da1d-4685-b288-0092732f0458-kube-api-access-rdpzv\") pod \"payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk\" (UID: \"78a8e8a6-da1d-4685-b288-0092732f0458\") " pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"
Jul 24 19:22:10.114681 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:10.114647    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:22:10.170928 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:10.170812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78a8e8a6-da1d-4685-b288-0092732f0458-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk\" (UID: \"78a8e8a6-da1d-4685-b288-0092732f0458\") " pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"
Jul 24 19:22:10.171462 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:10.171442    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78a8e8a6-da1d-4685-b288-0092732f0458-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk\" (UID: \"78a8e8a6-da1d-4685-b288-0092732f0458\") " pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"
Jul 24 19:22:10.171780 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:10.171761    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdpzv\" (UniqueName: \"kubernetes.io/projected/78a8e8a6-da1d-4685-b288-0092732f0458-kube-api-access-rdpzv\") pod \"payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk\" (UID: \"78a8e8a6-da1d-4685-b288-0092732f0458\") " pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"
Jul 24 19:22:10.181566 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:10.181535    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdpzv\" (UniqueName: \"kubernetes.io/projected/78a8e8a6-da1d-4685-b288-0092732f0458-kube-api-access-rdpzv\") pod \"payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk\" (UID: \"78a8e8a6-da1d-4685-b288-0092732f0458\") " pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"
Jul 24 19:22:10.250368 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:10.249262    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"]
Jul 24 19:22:10.277418 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:10.277376    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"
Jul 24 19:22:10.410325 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:10.410293    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"]
Jul 24 19:22:10.411776 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:22:10.411747    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a8e8a6_da1d_4685_b288_0092732f0458.slice/crio-937fdb98b1fe54b81e6b1bdfd424b6a49570a3b3df6eb3d0377b5e5e19fbd9a7 WatchSource:0}: Error finding container 937fdb98b1fe54b81e6b1bdfd424b6a49570a3b3df6eb3d0377b5e5e19fbd9a7: Status 404 returned error can't find the container with id 937fdb98b1fe54b81e6b1bdfd424b6a49570a3b3df6eb3d0377b5e5e19fbd9a7
Jul 24 19:22:11.186160 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:11.186050    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk" event={"ID":"78a8e8a6-da1d-4685-b288-0092732f0458","Type":"ContainerStarted","Data":"e769038001149990f09d4da9649604e6744542152586f5cb2ca3e323b86ae515"}
Jul 24 19:22:11.186160 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:11.186119    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"
Jul 24 19:22:11.186160 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:11.186145    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk" event={"ID":"78a8e8a6-da1d-4685-b288-0092732f0458","Type":"ContainerStarted","Data":"937fdb98b1fe54b81e6b1bdfd424b6a49570a3b3df6eb3d0377b5e5e19fbd9a7"}
Jul 24 19:22:11.187832 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:11.187806    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv" event={"ID":"842bf3a4-5e60-4045-a932-6f569740ee01","Type":"ContainerStarted","Data":"4a95632abc1277564b3f7b10f59bf4e5b8b2ddde7227c0bdbbe19efd5a013be1"}
Jul 24 19:22:11.187955 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:11.187849    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv" event={"ID":"842bf3a4-5e60-4045-a932-6f569740ee01","Type":"ContainerStarted","Data":"26cd046efae0d01b2162b3341ac1ed7e80efce3b28d79b349b72f428d1eac9af"}
Jul 24 19:22:11.187955 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:11.187897    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:22:11.202991 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:11.202943    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk" podStartSLOduration=2.202929571 podStartE2EDuration="2.202929571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:22:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:22:11.201824485 +0000 UTC m=+2269.592775964" watchObservedRunningTime="2026-07-24 19:22:11.202929571 +0000 UTC m=+2269.593881047"
Jul 24 19:22:11.220930 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:11.220882    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv" podStartSLOduration=1.636784478 podStartE2EDuration="2.220866945s" totalImagesPullingTime="584.082467ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:22:11.22004006 +0000 UTC m=+2269.610991539" watchObservedRunningTime="2026-07-24 19:22:11.220866945 +0000 UTC m=+2269.611818423"
Jul 24 19:22:17.054603 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.054563    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:22:17.198634 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.198603    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:22:17.250217 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.250184    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"]
Jul 24 19:22:17.253687 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.253666    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:22:17.256135 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.256112    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-48f4d2-serving-cert\""
Jul 24 19:22:17.263769 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.263734    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"]
Jul 24 19:22:17.335177 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.335142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/364046bf-1994-4881-a3bf-44d5837b73d4-maas-api-tls\") pod \"maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72\" (UID: \"364046bf-1994-4881-a3bf-44d5837b73d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:22:17.335369 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.335192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8xk2\" (UniqueName: \"kubernetes.io/projected/364046bf-1994-4881-a3bf-44d5837b73d4-kube-api-access-w8xk2\") pod \"maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72\" (UID: \"364046bf-1994-4881-a3bf-44d5837b73d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:22:17.335369 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.335222    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/364046bf-1994-4881-a3bf-44d5837b73d4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72\" (UID: \"364046bf-1994-4881-a3bf-44d5837b73d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:22:17.436634 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.436596    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/364046bf-1994-4881-a3bf-44d5837b73d4-maas-api-tls\") pod \"maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72\" (UID: \"364046bf-1994-4881-a3bf-44d5837b73d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:22:17.436840 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.436671    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8xk2\" (UniqueName: \"kubernetes.io/projected/364046bf-1994-4881-a3bf-44d5837b73d4-kube-api-access-w8xk2\") pod \"maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72\" (UID: \"364046bf-1994-4881-a3bf-44d5837b73d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:22:17.436840 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.436706    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/364046bf-1994-4881-a3bf-44d5837b73d4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72\" (UID: \"364046bf-1994-4881-a3bf-44d5837b73d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:22:17.437419 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.437359    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/364046bf-1994-4881-a3bf-44d5837b73d4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72\" (UID: \"364046bf-1994-4881-a3bf-44d5837b73d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:22:17.439236 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.439219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/364046bf-1994-4881-a3bf-44d5837b73d4-maas-api-tls\") pod \"maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72\" (UID: \"364046bf-1994-4881-a3bf-44d5837b73d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:22:17.444681 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.444663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8xk2\" (UniqueName: \"kubernetes.io/projected/364046bf-1994-4881-a3bf-44d5837b73d4-kube-api-access-w8xk2\") pod \"maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72\" (UID: \"364046bf-1994-4881-a3bf-44d5837b73d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:22:17.567907 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.567867    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:22:17.706309 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.706215    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"]
Jul 24 19:22:17.708832 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:22:17.708792    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod364046bf_1994_4881_a3bf_44d5837b73d4.slice/crio-f5dc10a7b43b3b37b6e28dc5b2927fab0559afc6766ab4693c20079575f49079 WatchSource:0}: Error finding container f5dc10a7b43b3b37b6e28dc5b2927fab0559afc6766ab4693c20079575f49079: Status 404 returned error can't find the container with id f5dc10a7b43b3b37b6e28dc5b2927fab0559afc6766ab4693c20079575f49079
Jul 24 19:22:17.936869 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.936786    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"]
Jul 24 19:22:17.941441 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.941419    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"
Jul 24 19:22:17.943760 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.943734    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-48f4d2-dockercfg-ntttx\""
Jul 24 19:22:17.943903 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.943740    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-48f4d2\""
Jul 24 19:22:17.951046 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:17.951023    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"]
Jul 24 19:22:18.042987 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:18.042954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x4z9\" (UniqueName: \"kubernetes.io/projected/6ab47fd9-2876-43be-95bc-a6a3294c268a-kube-api-access-9x4z9\") pod \"payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz\" (UID: \"6ab47fd9-2876-43be-95bc-a6a3294c268a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"
Jul 24 19:22:18.043266 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:18.043234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ab47fd9-2876-43be-95bc-a6a3294c268a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz\" (UID: \"6ab47fd9-2876-43be-95bc-a6a3294c268a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"
Jul 24 19:22:18.143708 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:18.143684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x4z9\" (UniqueName: \"kubernetes.io/projected/6ab47fd9-2876-43be-95bc-a6a3294c268a-kube-api-access-9x4z9\") pod \"payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz\" (UID: \"6ab47fd9-2876-43be-95bc-a6a3294c268a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"
Jul 24 19:22:18.144089 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:18.143720    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ab47fd9-2876-43be-95bc-a6a3294c268a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz\" (UID: \"6ab47fd9-2876-43be-95bc-a6a3294c268a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"
Jul 24 19:22:18.144484 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:18.144466    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ab47fd9-2876-43be-95bc-a6a3294c268a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz\" (UID: \"6ab47fd9-2876-43be-95bc-a6a3294c268a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"
Jul 24 19:22:18.152146 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:18.152124    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x4z9\" (UniqueName: \"kubernetes.io/projected/6ab47fd9-2876-43be-95bc-a6a3294c268a-kube-api-access-9x4z9\") pod \"payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz\" (UID: \"6ab47fd9-2876-43be-95bc-a6a3294c268a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"
Jul 24 19:22:18.234481 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:18.233973    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72" event={"ID":"364046bf-1994-4881-a3bf-44d5837b73d4","Type":"ContainerStarted","Data":"33684aeb12c5f00dc098b6f8c81e68c842139ab44cb3c6d233bdf6eb5caed8df"}
Jul 24 19:22:18.234481 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:18.234062    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72" event={"ID":"364046bf-1994-4881-a3bf-44d5837b73d4","Type":"ContainerStarted","Data":"f5dc10a7b43b3b37b6e28dc5b2927fab0559afc6766ab4693c20079575f49079"}
Jul 24 19:22:18.234882 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:18.234858    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:22:18.253364 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:18.253306    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72" podStartSLOduration=0.90099575 podStartE2EDuration="1.25329094s" totalImagesPullingTime="352.29519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:22:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:22:18.250945984 +0000 UTC m=+2276.641897471" watchObservedRunningTime="2026-07-24 19:22:18.25329094 +0000 UTC m=+2276.644242417"
Jul 24 19:22:18.256495 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:18.256476    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"
Jul 24 19:22:18.433830 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:18.433013    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"]
Jul 24 19:22:18.439859 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:22:18.439828    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ab47fd9_2876_43be_95bc_a6a3294c268a.slice/crio-1fa6697252a275d05f875e424fb942f3d0ee95c6bd4a74181e2a6dd83123f80b WatchSource:0}: Error finding container 1fa6697252a275d05f875e424fb942f3d0ee95c6bd4a74181e2a6dd83123f80b: Status 404 returned error can't find the container with id 1fa6697252a275d05f875e424fb942f3d0ee95c6bd4a74181e2a6dd83123f80b
Jul 24 19:22:19.242451 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:19.242415    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz" event={"ID":"6ab47fd9-2876-43be-95bc-a6a3294c268a","Type":"ContainerStarted","Data":"e1c933dc4c57780b43c9ac307d21a4e05ab49fa092ccdf6a072e62b4efa851d6"}
Jul 24 19:22:19.242922 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:19.242470    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz" event={"ID":"6ab47fd9-2876-43be-95bc-a6a3294c268a","Type":"ContainerStarted","Data":"1fa6697252a275d05f875e424fb942f3d0ee95c6bd4a74181e2a6dd83123f80b"}
Jul 24 19:22:19.242922 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:19.242599    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"
Jul 24 19:22:19.259566 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:19.259520    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz" podStartSLOduration=2.259505585 podStartE2EDuration="2.259505585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:22:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:22:19.257584283 +0000 UTC m=+2277.648535761" watchObservedRunningTime="2026-07-24 19:22:19.259505585 +0000 UTC m=+2277.650457063"
Jul 24 19:22:22.195555 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:22.195533    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"
Jul 24 19:22:24.209602 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:24.209554    2563 patch_prober.go:28] interesting pod/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 24 19:22:24.210061 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:24.209626    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw" podUID="91816a65-55c7-452d-ba71-26266c4908d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 24 19:22:24.248502 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:24.248464    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:22:25.275424 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:25.275286    2563 generic.go:356] "Generic (PLEG): container finished" podID="91816a65-55c7-452d-ba71-26266c4908d4" containerID="9795d2cc7a7950150689e90e8d3a3e2bb895bce91a172f6084e7a76c04f5c903" exitCode=0
Jul 24 19:22:25.275424 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:25.275367    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw" event={"ID":"91816a65-55c7-452d-ba71-26266c4908d4","Type":"ContainerDied","Data":"9795d2cc7a7950150689e90e8d3a3e2bb895bce91a172f6084e7a76c04f5c903"}
Jul 24 19:22:25.275999 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:25.275907    2563 scope.go:117] "RemoveContainer" containerID="9795d2cc7a7950150689e90e8d3a3e2bb895bce91a172f6084e7a76c04f5c903"
Jul 24 19:22:25.277060 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:25.277028    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:22:25.277175 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:25.277134    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:22:25.277303 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:25.277183    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 19:22:26.282180 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:26.282149    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw" event={"ID":"91816a65-55c7-452d-ba71-26266c4908d4","Type":"ContainerStarted","Data":"de2929d0cc8636b83d341a6b883ec2f68536f1dc17e03fb01729ec0ccea9390a"}
Jul 24 19:22:26.981895 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:26.981859    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:26.982083 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:26.982037    2563 patch_prober.go:28] interesting pod/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 24 19:22:26.982135 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:26.982082    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw" podUID="91816a65-55c7-452d-ba71-26266c4908d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 24 19:22:27.860887 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:27.860851    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:22:27.982860 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:27.982824    2563 patch_prober.go:28] interesting pod/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 24 19:22:27.983036 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:27.982882    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw" podUID="91816a65-55c7-452d-ba71-26266c4908d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 24 19:22:28.982202 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:28.982167    2563 patch_prober.go:28] interesting pod/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 24 19:22:28.982659 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:28.982218    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw" podUID="91816a65-55c7-452d-ba71-26266c4908d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 24 19:22:30.033017 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:30.032972    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:30.034345 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:30.034324    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:30.036351 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:30.036318    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:22:30.249003 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:30.248966    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz" podUID="6ab47fd9-2876-43be-95bc-a6a3294c268a" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.64:9004: connect: connection refused"
Jul 24 19:22:37.459971 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:37.459930    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:22:40.248073 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:22:40.248042    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"
Jul 24 19:23:10.474988 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.474952    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"]
Jul 24 19:23:10.478355 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.478338    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.481683 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.481660    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-48f4d2\"/\"default-dockercfg-xgbkb\""
Jul 24 19:23:10.481814 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.481713    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-48f4d2\"/\"openshift-service-ca.crt\""
Jul 24 19:23:10.481814 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.481723    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-48f4d2\"/\"auth-test-model-48f4d2-kserve-self-signed-certs\""
Jul 24 19:23:10.481814 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.481772    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-48f4d2\"/\"kube-root-ca.crt\""
Jul 24 19:23:10.488167 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.488143    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"]
Jul 24 19:23:10.608323 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.608292    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-dshm\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.608549 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.608330    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-tls-certs\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.608549 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.608357    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-home\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.608549 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.608485    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-model-cache\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.608549 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.608513    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-tmp-dir\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.608736 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.608564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngcgk\" (UniqueName: \"kubernetes.io/projected/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-kube-api-access-ngcgk\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.709861 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.709828    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-dshm\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.710069 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.709869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-tls-certs\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.710069 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.709898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-home\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.710069 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.709946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-model-cache\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.710069 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.709971    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-tmp-dir\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.710069 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.709994    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngcgk\" (UniqueName: \"kubernetes.io/projected/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-kube-api-access-ngcgk\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.710412 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.710370    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-home\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.710505 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.710420    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-model-cache\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.710699 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.710600    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-tmp-dir\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.712269 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.712250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-dshm\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.712768 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.712750    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-tls-certs\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.718180 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.718158    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngcgk\" (UniqueName: \"kubernetes.io/projected/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-kube-api-access-ngcgk\") pod \"auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") " pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.788309 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.788250    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:10.943160 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:10.943134    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"]
Jul 24 19:23:10.945296 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:23:10.945252    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74f7bfe0_9f8b_4fda_b8f1_b2de257f1528.slice/crio-f3466ab5517c1b7e589f9b8fd85f945ffc01ed5683b39f9d84ac0639a4925bc4 WatchSource:0}: Error finding container f3466ab5517c1b7e589f9b8fd85f945ffc01ed5683b39f9d84ac0639a4925bc4: Status 404 returned error can't find the container with id f3466ab5517c1b7e589f9b8fd85f945ffc01ed5683b39f9d84ac0639a4925bc4
Jul 24 19:23:11.514157 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:11.514120    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq" event={"ID":"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528","Type":"ContainerStarted","Data":"530ba8307d51f7e0642188349067f3e3ed2de7e7e1602f8e041fe506f76718b4"}
Jul 24 19:23:11.514157 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:11.514167    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq" event={"ID":"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528","Type":"ContainerStarted","Data":"f3466ab5517c1b7e589f9b8fd85f945ffc01ed5683b39f9d84ac0639a4925bc4"}
Jul 24 19:23:11.532249 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:11.532204    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq" podStartSLOduration=1.532190647 podStartE2EDuration="1.532190647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:23:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:23:11.529261315 +0000 UTC m=+2329.920212791" watchObservedRunningTime="2026-07-24 19:23:11.532190647 +0000 UTC m=+2329.923142135"
Jul 24 19:23:14.156258 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:14.156159    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:23:20.788374 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:20.788341    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:20.788880 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:20.788411    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:20.800973 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:20.800945    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:21.577997 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:21.577960    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:23:38.848806 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:38.848762    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:23:52.653362 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:23:52.653315    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:24:08.046549 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:24:08.046504    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:24:22.264446 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:24:22.264414    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 19:24:22.272682 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:24:22.272661    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 19:24:27.558655 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:24:27.558622    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:24:46.350552 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:24:46.350516    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:25:02.549991 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:02.549954    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:25:18.178854 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:18.178813    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"]
Jul 24 19:25:20.848839 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:20.848790    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:25:33.198428 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.198372    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq" podUID="74f7bfe0-9f8b-4fda-b8f1-b2de257f1528" containerName="main" containerID="cri-o://530ba8307d51f7e0642188349067f3e3ed2de7e7e1602f8e041fe506f76718b4" gracePeriod=45
Jul 24 19:25:33.450568 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.450508    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:25:33.557457 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.557416    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-dshm\") pod \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") "
Jul 24 19:25:33.557664 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.557497    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-home\") pod \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") "
Jul 24 19:25:33.557664 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.557527    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-tls-certs\") pod \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") "
Jul 24 19:25:33.557664 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.557594    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-model-cache\") pod \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") "
Jul 24 19:25:33.557664 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.557621    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-tmp-dir\") pod \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") "
Jul 24 19:25:33.557664 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.557650    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngcgk\" (UniqueName: \"kubernetes.io/projected/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-kube-api-access-ngcgk\") pod \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\" (UID: \"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528\") "
Jul 24 19:25:33.557965 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.557809    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-home" (OuterVolumeSpecName: "home") pod "74f7bfe0-9f8b-4fda-b8f1-b2de257f1528" (UID: "74f7bfe0-9f8b-4fda-b8f1-b2de257f1528"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:25:33.557965 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.557893    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-model-cache" (OuterVolumeSpecName: "model-cache") pod "74f7bfe0-9f8b-4fda-b8f1-b2de257f1528" (UID: "74f7bfe0-9f8b-4fda-b8f1-b2de257f1528"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:25:33.557965 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.557904    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "74f7bfe0-9f8b-4fda-b8f1-b2de257f1528" (UID: "74f7bfe0-9f8b-4fda-b8f1-b2de257f1528"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:25:33.557965 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.557954    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-home\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:25:33.559962 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.559929    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-dshm" (OuterVolumeSpecName: "dshm") pod "74f7bfe0-9f8b-4fda-b8f1-b2de257f1528" (UID: "74f7bfe0-9f8b-4fda-b8f1-b2de257f1528"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:25:33.559962 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.559953    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "74f7bfe0-9f8b-4fda-b8f1-b2de257f1528" (UID: "74f7bfe0-9f8b-4fda-b8f1-b2de257f1528"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:25:33.560105 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.560001    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-kube-api-access-ngcgk" (OuterVolumeSpecName: "kube-api-access-ngcgk") pod "74f7bfe0-9f8b-4fda-b8f1-b2de257f1528" (UID: "74f7bfe0-9f8b-4fda-b8f1-b2de257f1528"). InnerVolumeSpecName "kube-api-access-ngcgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:25:33.658618 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.658569    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-dshm\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:25:33.658618 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.658613    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-tls-certs\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:25:33.658618 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.658625    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-model-cache\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:25:33.658618 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.658632    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-tmp-dir\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:25:33.658899 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:33.658642    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngcgk\" (UniqueName: \"kubernetes.io/projected/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528-kube-api-access-ngcgk\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:25:34.247021 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:34.246985    2563 generic.go:356] "Generic (PLEG): container finished" podID="74f7bfe0-9f8b-4fda-b8f1-b2de257f1528" containerID="530ba8307d51f7e0642188349067f3e3ed2de7e7e1602f8e041fe506f76718b4" exitCode=0
Jul 24 19:25:34.247021 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:34.247029    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"
Jul 24 19:25:34.247519 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:34.247068    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq" event={"ID":"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528","Type":"ContainerDied","Data":"530ba8307d51f7e0642188349067f3e3ed2de7e7e1602f8e041fe506f76718b4"}
Jul 24 19:25:34.247519 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:34.247118    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq" event={"ID":"74f7bfe0-9f8b-4fda-b8f1-b2de257f1528","Type":"ContainerDied","Data":"f3466ab5517c1b7e589f9b8fd85f945ffc01ed5683b39f9d84ac0639a4925bc4"}
Jul 24 19:25:34.247519 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:34.247136    2563 scope.go:117] "RemoveContainer" containerID="530ba8307d51f7e0642188349067f3e3ed2de7e7e1602f8e041fe506f76718b4"
Jul 24 19:25:34.256281 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:34.256264    2563 scope.go:117] "RemoveContainer" containerID="530ba8307d51f7e0642188349067f3e3ed2de7e7e1602f8e041fe506f76718b4"
Jul 24 19:25:34.256578 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:25:34.256559    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"530ba8307d51f7e0642188349067f3e3ed2de7e7e1602f8e041fe506f76718b4\": container with ID starting with 530ba8307d51f7e0642188349067f3e3ed2de7e7e1602f8e041fe506f76718b4 not found: ID does not exist" containerID="530ba8307d51f7e0642188349067f3e3ed2de7e7e1602f8e041fe506f76718b4"
Jul 24 19:25:34.256632 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:34.256586    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"530ba8307d51f7e0642188349067f3e3ed2de7e7e1602f8e041fe506f76718b4"} err="failed to get container status \"530ba8307d51f7e0642188349067f3e3ed2de7e7e1602f8e041fe506f76718b4\": rpc error: code = NotFound desc = could not find container \"530ba8307d51f7e0642188349067f3e3ed2de7e7e1602f8e041fe506f76718b4\": container with ID starting with 530ba8307d51f7e0642188349067f3e3ed2de7e7e1602f8e041fe506f76718b4 not found: ID does not exist"
Jul 24 19:25:34.262546 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:34.262524    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"]
Jul 24 19:25:34.269153 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:34.269129    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-48f4d2/auth-test-model-48f4d2-kserve-7dd44bb467-fhwhq"]
Jul 24 19:25:36.204673 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:36.204637    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74f7bfe0-9f8b-4fda-b8f1-b2de257f1528" path="/var/lib/kubelet/pods/74f7bfe0-9f8b-4fda-b8f1-b2de257f1528/volumes"
Jul 24 19:25:37.251248 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:25:37.251214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:26:06.651020 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:06.650979    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:26:20.958928 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:20.958848    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:26:41.252577 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:41.252541    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:26:42.950569 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.950537    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"]
Jul 24 19:26:42.950968 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.950854    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74f7bfe0-9f8b-4fda-b8f1-b2de257f1528" containerName="main"
Jul 24 19:26:42.950968 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.950864    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="74f7bfe0-9f8b-4fda-b8f1-b2de257f1528" containerName="main"
Jul 24 19:26:42.950968 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.950931    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="74f7bfe0-9f8b-4fda-b8f1-b2de257f1528" containerName="main"
Jul 24 19:26:42.953874 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.953850    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:42.956605 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.956583    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e05f4d\"/\"kube-root-ca.crt\""
Jul 24 19:26:42.957595 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.957569    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e05f4d\"/\"default-dockercfg-v29xv\""
Jul 24 19:26:42.957736 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.957719    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e05f4d\"/\"openshift-service-ca.crt\""
Jul 24 19:26:42.957838 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.957824    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e05f4d\"/\"rate-test-model-e05f4d-kserve-self-signed-certs\""
Jul 24 19:26:42.965142 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.965117    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"]
Jul 24 19:26:42.971044 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.971024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/821fb1ff-bc69-4297-a827-bcd8d651ec12-tls-certs\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:42.971153 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.971060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-model-cache\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:42.971153 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.971088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-dshm\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:42.971153 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.971119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-home\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:42.971320 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.971192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8sw4\" (UniqueName: \"kubernetes.io/projected/821fb1ff-bc69-4297-a827-bcd8d651ec12-kube-api-access-q8sw4\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:42.971320 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:42.971248    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-tmp-dir\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:43.072572 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.072533    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/821fb1ff-bc69-4297-a827-bcd8d651ec12-tls-certs\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:43.072759 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.072580    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-model-cache\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:43.072759 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.072710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-dshm\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:43.072759 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.072754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-home\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:43.072934 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.072805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8sw4\" (UniqueName: \"kubernetes.io/projected/821fb1ff-bc69-4297-a827-bcd8d651ec12-kube-api-access-q8sw4\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:43.072934 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.072853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-tmp-dir\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:43.073028 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.072985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-model-cache\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:43.073186 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.073160    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-home\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:43.073520 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.073197    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-tmp-dir\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:43.075018 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.074996    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-dshm\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:43.075633 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.075612    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/821fb1ff-bc69-4297-a827-bcd8d651ec12-tls-certs\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:43.081341 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.081313    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8sw4\" (UniqueName: \"kubernetes.io/projected/821fb1ff-bc69-4297-a827-bcd8d651ec12-kube-api-access-q8sw4\") pod \"rate-test-model-e05f4d-kserve-859dd56f55-5cmph\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") " pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:43.263335 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.263256    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:43.398861 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.398838    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"]
Jul 24 19:26:43.400449 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:26:43.400422    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod821fb1ff_bc69_4297_a827_bcd8d651ec12.slice/crio-f43fa1f5db7e9098c211d887ab4c61bc10ec05c06421d24ca43f0a1aa9e1c49f WatchSource:0}: Error finding container f43fa1f5db7e9098c211d887ab4c61bc10ec05c06421d24ca43f0a1aa9e1c49f: Status 404 returned error can't find the container with id f43fa1f5db7e9098c211d887ab4c61bc10ec05c06421d24ca43f0a1aa9e1c49f
Jul 24 19:26:43.609831 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.609786    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph" event={"ID":"821fb1ff-bc69-4297-a827-bcd8d651ec12","Type":"ContainerStarted","Data":"1f09f2025c1f4d0a467cc165e2cd9a932a9cf1606b9d9df7a9bc2b2989dfeca7"}
Jul 24 19:26:43.610017 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.609839    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph" event={"ID":"821fb1ff-bc69-4297-a827-bcd8d651ec12","Type":"ContainerStarted","Data":"f43fa1f5db7e9098c211d887ab4c61bc10ec05c06421d24ca43f0a1aa9e1c49f"}
Jul 24 19:26:43.627337 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:43.627289    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph" podStartSLOduration=1.627273848 podStartE2EDuration="1.627273848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:26:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:26:43.624983038 +0000 UTC m=+2542.015934706" watchObservedRunningTime="2026-07-24 19:26:43.627273848 +0000 UTC m=+2542.018225357"
Jul 24 19:26:53.263643 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:53.263604    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:53.264104 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:53.263818    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:53.276186 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:53.276164    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:26:53.347941 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:53.347904    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:26:53.669999 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:26:53.669966    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:27:04.048207 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:27:04.048168    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:27:22.438074 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:27:22.438039    2563 scope.go:117] "RemoveContainer" containerID="02e96e30f1ef1f4b030b1534177e550e2090abfd062a5fa7ef812359828bac5d"
Jul 24 19:27:32.059073 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:27:32.059039    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:27:47.246793 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:27:47.246756    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:27:52.598746 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:27:52.598710    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"]
Jul 24 19:28:04.550322 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:04.550284    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:28:07.619416 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.619331    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph" podUID="821fb1ff-bc69-4297-a827-bcd8d651ec12" containerName="main" containerID="cri-o://1f09f2025c1f4d0a467cc165e2cd9a932a9cf1606b9d9df7a9bc2b2989dfeca7" gracePeriod=45
Jul 24 19:28:07.864361 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.864336    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:28:07.923374 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.923289    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-model-cache\") pod \"821fb1ff-bc69-4297-a827-bcd8d651ec12\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") "
Jul 24 19:28:07.923374 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.923348    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8sw4\" (UniqueName: \"kubernetes.io/projected/821fb1ff-bc69-4297-a827-bcd8d651ec12-kube-api-access-q8sw4\") pod \"821fb1ff-bc69-4297-a827-bcd8d651ec12\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") "
Jul 24 19:28:07.923374 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.923372    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-dshm\") pod \"821fb1ff-bc69-4297-a827-bcd8d651ec12\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") "
Jul 24 19:28:07.923686 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.923441    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-tmp-dir\") pod \"821fb1ff-bc69-4297-a827-bcd8d651ec12\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") "
Jul 24 19:28:07.923686 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.923482    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-home\") pod \"821fb1ff-bc69-4297-a827-bcd8d651ec12\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") "
Jul 24 19:28:07.923686 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.923517    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/821fb1ff-bc69-4297-a827-bcd8d651ec12-tls-certs\") pod \"821fb1ff-bc69-4297-a827-bcd8d651ec12\" (UID: \"821fb1ff-bc69-4297-a827-bcd8d651ec12\") "
Jul 24 19:28:07.923686 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.923648    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-model-cache" (OuterVolumeSpecName: "model-cache") pod "821fb1ff-bc69-4297-a827-bcd8d651ec12" (UID: "821fb1ff-bc69-4297-a827-bcd8d651ec12"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:28:07.923901 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.923705    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "821fb1ff-bc69-4297-a827-bcd8d651ec12" (UID: "821fb1ff-bc69-4297-a827-bcd8d651ec12"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:28:07.923901 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.923718    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-home" (OuterVolumeSpecName: "home") pod "821fb1ff-bc69-4297-a827-bcd8d651ec12" (UID: "821fb1ff-bc69-4297-a827-bcd8d651ec12"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:28:07.923901 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.923825    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-tmp-dir\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:28:07.923901 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.923838    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-home\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:28:07.923901 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.923847    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-model-cache\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:28:07.925776 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.925746    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/821fb1ff-bc69-4297-a827-bcd8d651ec12-kube-api-access-q8sw4" (OuterVolumeSpecName: "kube-api-access-q8sw4") pod "821fb1ff-bc69-4297-a827-bcd8d651ec12" (UID: "821fb1ff-bc69-4297-a827-bcd8d651ec12"). InnerVolumeSpecName "kube-api-access-q8sw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:28:07.925884 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.925794    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/821fb1ff-bc69-4297-a827-bcd8d651ec12-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "821fb1ff-bc69-4297-a827-bcd8d651ec12" (UID: "821fb1ff-bc69-4297-a827-bcd8d651ec12"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:28:07.925884 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:07.925804    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-dshm" (OuterVolumeSpecName: "dshm") pod "821fb1ff-bc69-4297-a827-bcd8d651ec12" (UID: "821fb1ff-bc69-4297-a827-bcd8d651ec12"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:28:08.024843 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:08.024811    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/821fb1ff-bc69-4297-a827-bcd8d651ec12-tls-certs\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:28:08.024843 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:08.024837    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8sw4\" (UniqueName: \"kubernetes.io/projected/821fb1ff-bc69-4297-a827-bcd8d651ec12-kube-api-access-q8sw4\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:28:08.024843 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:08.024846    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/821fb1ff-bc69-4297-a827-bcd8d651ec12-dshm\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:28:08.039360 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:08.039329    2563 generic.go:356] "Generic (PLEG): container finished" podID="821fb1ff-bc69-4297-a827-bcd8d651ec12" containerID="1f09f2025c1f4d0a467cc165e2cd9a932a9cf1606b9d9df7a9bc2b2989dfeca7" exitCode=0
Jul 24 19:28:08.039526 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:08.039373    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"
Jul 24 19:28:08.039526 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:08.039409    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph" event={"ID":"821fb1ff-bc69-4297-a827-bcd8d651ec12","Type":"ContainerDied","Data":"1f09f2025c1f4d0a467cc165e2cd9a932a9cf1606b9d9df7a9bc2b2989dfeca7"}
Jul 24 19:28:08.039526 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:08.039451    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph" event={"ID":"821fb1ff-bc69-4297-a827-bcd8d651ec12","Type":"ContainerDied","Data":"f43fa1f5db7e9098c211d887ab4c61bc10ec05c06421d24ca43f0a1aa9e1c49f"}
Jul 24 19:28:08.039526 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:08.039469    2563 scope.go:117] "RemoveContainer" containerID="1f09f2025c1f4d0a467cc165e2cd9a932a9cf1606b9d9df7a9bc2b2989dfeca7"
Jul 24 19:28:08.050943 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:08.050924    2563 scope.go:117] "RemoveContainer" containerID="1f09f2025c1f4d0a467cc165e2cd9a932a9cf1606b9d9df7a9bc2b2989dfeca7"
Jul 24 19:28:08.051185 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:28:08.051168    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f09f2025c1f4d0a467cc165e2cd9a932a9cf1606b9d9df7a9bc2b2989dfeca7\": container with ID starting with 1f09f2025c1f4d0a467cc165e2cd9a932a9cf1606b9d9df7a9bc2b2989dfeca7 not found: ID does not exist" containerID="1f09f2025c1f4d0a467cc165e2cd9a932a9cf1606b9d9df7a9bc2b2989dfeca7"
Jul 24 19:28:08.051240 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:08.051200    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f09f2025c1f4d0a467cc165e2cd9a932a9cf1606b9d9df7a9bc2b2989dfeca7"} err="failed to get container status \"1f09f2025c1f4d0a467cc165e2cd9a932a9cf1606b9d9df7a9bc2b2989dfeca7\": rpc error: code = NotFound desc = could not find container \"1f09f2025c1f4d0a467cc165e2cd9a932a9cf1606b9d9df7a9bc2b2989dfeca7\": container with ID starting with 1f09f2025c1f4d0a467cc165e2cd9a932a9cf1606b9d9df7a9bc2b2989dfeca7 not found: ID does not exist"
Jul 24 19:28:08.063786 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:08.063762    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"]
Jul 24 19:28:08.069841 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:08.069818    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e05f4d/rate-test-model-e05f4d-kserve-859dd56f55-5cmph"]
Jul 24 19:28:08.204278 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:08.204197    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="821fb1ff-bc69-4297-a827-bcd8d651ec12" path="/var/lib/kubelet/pods/821fb1ff-bc69-4297-a827-bcd8d651ec12/volumes"
Jul 24 19:28:17.267383 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.267343    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"]
Jul 24 19:28:17.267854 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.267719    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="821fb1ff-bc69-4297-a827-bcd8d651ec12" containerName="main"
Jul 24 19:28:17.267854 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.267733    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="821fb1ff-bc69-4297-a827-bcd8d651ec12" containerName="main"
Jul 24 19:28:17.267854 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.267788    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="821fb1ff-bc69-4297-a827-bcd8d651ec12" containerName="main"
Jul 24 19:28:17.272138 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.272116    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:28:17.275368 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.275348    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-87cd66-serving-cert\""
Jul 24 19:28:17.284584 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.284558    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"]
Jul 24 19:28:17.304414 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.304352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-maas-api-tls\") pod \"maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc\" (UID: \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:28:17.304582 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.304471    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc\" (UID: \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:28:17.304633 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.304572    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tczk\" (UniqueName: \"kubernetes.io/projected/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-kube-api-access-4tczk\") pod \"maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc\" (UID: \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:28:17.405465 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.405430    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc\" (UID: \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:28:17.405654 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.405498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tczk\" (UniqueName: \"kubernetes.io/projected/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-kube-api-access-4tczk\") pod \"maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc\" (UID: \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:28:17.405654 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.405535    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-maas-api-tls\") pod \"maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc\" (UID: \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:28:17.405654 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:28:17.405635    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-serving-cert: secret "maas-api-e2e-ipp-a-87cd66-serving-cert" not found
Jul 24 19:28:17.405786 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:28:17.405703    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-maas-api-tls podName:fa0f22d5-9afe-4d04-bbe8-b63075f9bd85 nodeName:}" failed. No retries permitted until 2026-07-24 19:28:17.905686774 +0000 UTC m=+2636.296638230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-maas-api-tls") pod "maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc" (UID: "fa0f22d5-9afe-4d04-bbe8-b63075f9bd85") : secret "maas-api-e2e-ipp-a-87cd66-serving-cert" not found
Jul 24 19:28:17.406218 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.406194    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc\" (UID: \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:28:17.419747 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.419715    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tczk\" (UniqueName: \"kubernetes.io/projected/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-kube-api-access-4tczk\") pod \"maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc\" (UID: \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:28:17.907664 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.907627    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"]
Jul 24 19:28:17.910652 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.910624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-maas-api-tls\") pod \"maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc\" (UID: \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:28:17.911123 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.911107    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"
Jul 24 19:28:17.913326 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.913299    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-maas-api-tls\") pod \"maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc\" (UID: \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:28:17.913629 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.913612    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-87cd66\""
Jul 24 19:28:17.913702 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.913652    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-87cd66-dockercfg-sqsdw\""
Jul 24 19:28:17.922636 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.922609    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"]
Jul 24 19:28:17.933829 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.933805    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"]
Jul 24 19:28:17.937124 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.937105    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"
Jul 24 19:28:17.949866 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:17.949843    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"]
Jul 24 19:28:18.011973 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.011932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bd845bf-485c-4ce3-8d89-79b7a99dcf71-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52\" (UID: \"2bd845bf-485c-4ce3-8d89-79b7a99dcf71\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"
Jul 24 19:28:18.012170 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.012037    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hbf7\" (UniqueName: \"kubernetes.io/projected/2bd845bf-485c-4ce3-8d89-79b7a99dcf71-kube-api-access-4hbf7\") pod \"payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52\" (UID: \"2bd845bf-485c-4ce3-8d89-79b7a99dcf71\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"
Jul 24 19:28:18.012170 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.012064    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d67k\" (UniqueName: \"kubernetes.io/projected/b0d0bdef-5d75-4a72-8132-c2b076f17a85-kube-api-access-8d67k\") pod \"payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t\" (UID: \"b0d0bdef-5d75-4a72-8132-c2b076f17a85\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"
Jul 24 19:28:18.012170 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.012092    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0d0bdef-5d75-4a72-8132-c2b076f17a85-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t\" (UID: \"b0d0bdef-5d75-4a72-8132-c2b076f17a85\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"
Jul 24 19:28:18.113199 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.113170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bd845bf-485c-4ce3-8d89-79b7a99dcf71-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52\" (UID: \"2bd845bf-485c-4ce3-8d89-79b7a99dcf71\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"
Jul 24 19:28:18.113450 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.113225    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hbf7\" (UniqueName: \"kubernetes.io/projected/2bd845bf-485c-4ce3-8d89-79b7a99dcf71-kube-api-access-4hbf7\") pod \"payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52\" (UID: \"2bd845bf-485c-4ce3-8d89-79b7a99dcf71\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"
Jul 24 19:28:18.113450 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.113247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d67k\" (UniqueName: \"kubernetes.io/projected/b0d0bdef-5d75-4a72-8132-c2b076f17a85-kube-api-access-8d67k\") pod \"payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t\" (UID: \"b0d0bdef-5d75-4a72-8132-c2b076f17a85\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"
Jul 24 19:28:18.113450 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.113266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0d0bdef-5d75-4a72-8132-c2b076f17a85-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t\" (UID: \"b0d0bdef-5d75-4a72-8132-c2b076f17a85\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"
Jul 24 19:28:18.113928 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.113908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bd845bf-485c-4ce3-8d89-79b7a99dcf71-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52\" (UID: \"2bd845bf-485c-4ce3-8d89-79b7a99dcf71\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"
Jul 24 19:28:18.114006 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.113988    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0d0bdef-5d75-4a72-8132-c2b076f17a85-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t\" (UID: \"b0d0bdef-5d75-4a72-8132-c2b076f17a85\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"
Jul 24 19:28:18.121668 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.121644    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hbf7\" (UniqueName: \"kubernetes.io/projected/2bd845bf-485c-4ce3-8d89-79b7a99dcf71-kube-api-access-4hbf7\") pod \"payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52\" (UID: \"2bd845bf-485c-4ce3-8d89-79b7a99dcf71\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"
Jul 24 19:28:18.121851 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.121828    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d67k\" (UniqueName: \"kubernetes.io/projected/b0d0bdef-5d75-4a72-8132-c2b076f17a85-kube-api-access-8d67k\") pod \"payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t\" (UID: \"b0d0bdef-5d75-4a72-8132-c2b076f17a85\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"
Jul 24 19:28:18.182791 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.182714    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:28:18.231051 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.230989    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"
Jul 24 19:28:18.247763 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.247712    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"
Jul 24 19:28:18.339586 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.339558    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"]
Jul 24 19:28:18.342102 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:28:18.342075    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa0f22d5_9afe_4d04_bbe8_b63075f9bd85.slice/crio-cfeb605e97c654c261cd8a4932f5a4c09e387a6f05e9b812355364c5e588b595 WatchSource:0}: Error finding container cfeb605e97c654c261cd8a4932f5a4c09e387a6f05e9b812355364c5e588b595: Status 404 returned error can't find the container with id cfeb605e97c654c261cd8a4932f5a4c09e387a6f05e9b812355364c5e588b595
Jul 24 19:28:18.343717 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.343090    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 19:28:18.420135 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.419984    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"]
Jul 24 19:28:18.422759 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:28:18.422727    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd845bf_485c_4ce3_8d89_79b7a99dcf71.slice/crio-2621de9fb9dcc4e2d3a8afa4887ae1ded5be0970b5edaae94d58f8e3c68d8cb3 WatchSource:0}: Error finding container 2621de9fb9dcc4e2d3a8afa4887ae1ded5be0970b5edaae94d58f8e3c68d8cb3: Status 404 returned error can't find the container with id 2621de9fb9dcc4e2d3a8afa4887ae1ded5be0970b5edaae94d58f8e3c68d8cb3
Jul 24 19:28:18.615106 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:18.615081    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"]
Jul 24 19:28:18.617410 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:28:18.617368    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0d0bdef_5d75_4a72_8132_c2b076f17a85.slice/crio-020fd39ff7f9a37a9a4582356b9343741b04bcbb994bf0d722501a45dacfbcc8 WatchSource:0}: Error finding container 020fd39ff7f9a37a9a4582356b9343741b04bcbb994bf0d722501a45dacfbcc8: Status 404 returned error can't find the container with id 020fd39ff7f9a37a9a4582356b9343741b04bcbb994bf0d722501a45dacfbcc8
Jul 24 19:28:19.097879 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:19.097840    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc" event={"ID":"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85","Type":"ContainerStarted","Data":"1a4175910ccd6e38ab7238a5b85aeffe519200301019060d5bf68c91913b1bad"}
Jul 24 19:28:19.098070 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:19.097911    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc" event={"ID":"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85","Type":"ContainerStarted","Data":"cfeb605e97c654c261cd8a4932f5a4c09e387a6f05e9b812355364c5e588b595"}
Jul 24 19:28:19.098070 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:19.097938    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:28:19.099438 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:19.099417    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52" event={"ID":"2bd845bf-485c-4ce3-8d89-79b7a99dcf71","Type":"ContainerStarted","Data":"0bac2c475bf5e229d080251393bf7483c09ea28f07d1e2345f13a58de0db4839"}
Jul 24 19:28:19.099559 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:19.099454    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52" event={"ID":"2bd845bf-485c-4ce3-8d89-79b7a99dcf71","Type":"ContainerStarted","Data":"2621de9fb9dcc4e2d3a8afa4887ae1ded5be0970b5edaae94d58f8e3c68d8cb3"}
Jul 24 19:28:19.099559 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:19.099496    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"
Jul 24 19:28:19.100922 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:19.100898    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t" event={"ID":"b0d0bdef-5d75-4a72-8132-c2b076f17a85","Type":"ContainerStarted","Data":"4d912c7618181565e523c149f50ce463798267b9abab4831874d6a8d2fa7f832"}
Jul 24 19:28:19.101024 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:19.100942    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t" event={"ID":"b0d0bdef-5d75-4a72-8132-c2b076f17a85","Type":"ContainerStarted","Data":"020fd39ff7f9a37a9a4582356b9343741b04bcbb994bf0d722501a45dacfbcc8"}
Jul 24 19:28:19.101085 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:19.101031    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"
Jul 24 19:28:19.116574 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:19.116533    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc" podStartSLOduration=1.690456865 podStartE2EDuration="2.116516735s" totalImagesPullingTime="426.05987ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:28:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:28:19.114240955 +0000 UTC m=+2637.505192474" watchObservedRunningTime="2026-07-24 19:28:19.116516735 +0000 UTC m=+2637.507468246"
Jul 24 19:28:19.130896 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:19.130857    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t" podStartSLOduration=2.130844035 podStartE2EDuration="2.130844035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:28:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:28:19.129529942 +0000 UTC m=+2637.520481419" watchObservedRunningTime="2026-07-24 19:28:19.130844035 +0000 UTC m=+2637.521795512"
Jul 24 19:28:19.145847 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:19.145808    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52" podStartSLOduration=2.14579365 podStartE2EDuration="2.14579365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:28:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:28:19.144723431 +0000 UTC m=+2637.535674910" watchObservedRunningTime="2026-07-24 19:28:19.14579365 +0000 UTC m=+2637.536745138"
Jul 24 19:28:25.113199 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:25.113171    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:28:25.349238 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:25.349196    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:28:30.108519 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:30.108487    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"
Jul 24 19:28:30.108935 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:30.108560    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"
Jul 24 19:28:36.204356 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.204319    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"]
Jul 24 19:28:36.207731 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.207714    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:28:36.210165 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.210140    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-5ce07c-serving-cert\""
Jul 24 19:28:36.218229 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.218204    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"]
Jul 24 19:28:36.278760 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.278727    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4124d1fa-edba-4079-a170-b30c203f545e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n\" (UID: \"4124d1fa-edba-4079-a170-b30c203f545e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:28:36.278986 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.278795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4124d1fa-edba-4079-a170-b30c203f545e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n\" (UID: \"4124d1fa-edba-4079-a170-b30c203f545e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:28:36.278986 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.278940    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkp7w\" (UniqueName: \"kubernetes.io/projected/4124d1fa-edba-4079-a170-b30c203f545e-kube-api-access-qkp7w\") pod \"maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n\" (UID: \"4124d1fa-edba-4079-a170-b30c203f545e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:28:36.380275 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.380240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkp7w\" (UniqueName: \"kubernetes.io/projected/4124d1fa-edba-4079-a170-b30c203f545e-kube-api-access-qkp7w\") pod \"maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n\" (UID: \"4124d1fa-edba-4079-a170-b30c203f545e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:28:36.380275 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.380275    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4124d1fa-edba-4079-a170-b30c203f545e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n\" (UID: \"4124d1fa-edba-4079-a170-b30c203f545e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:28:36.380578 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.380326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4124d1fa-edba-4079-a170-b30c203f545e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n\" (UID: \"4124d1fa-edba-4079-a170-b30c203f545e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:28:36.380578 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:28:36.380486    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-serving-cert: secret "maas-api-e2e-ipp-b-5ce07c-serving-cert" not found
Jul 24 19:28:36.380578 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:28:36.380554    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4124d1fa-edba-4079-a170-b30c203f545e-maas-api-tls podName:4124d1fa-edba-4079-a170-b30c203f545e nodeName:}" failed. No retries permitted until 2026-07-24 19:28:36.880534579 +0000 UTC m=+2655.271486033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4124d1fa-edba-4079-a170-b30c203f545e-maas-api-tls") pod "maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n" (UID: "4124d1fa-edba-4079-a170-b30c203f545e") : secret "maas-api-e2e-ipp-b-5ce07c-serving-cert" not found
Jul 24 19:28:36.381038 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.381019    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4124d1fa-edba-4079-a170-b30c203f545e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n\" (UID: \"4124d1fa-edba-4079-a170-b30c203f545e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:28:36.391260 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.391228    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkp7w\" (UniqueName: \"kubernetes.io/projected/4124d1fa-edba-4079-a170-b30c203f545e-kube-api-access-qkp7w\") pod \"maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n\" (UID: \"4124d1fa-edba-4079-a170-b30c203f545e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:28:36.885348 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.885310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4124d1fa-edba-4079-a170-b30c203f545e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n\" (UID: \"4124d1fa-edba-4079-a170-b30c203f545e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:28:36.887964 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.887942    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4124d1fa-edba-4079-a170-b30c203f545e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n\" (UID: \"4124d1fa-edba-4079-a170-b30c203f545e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:28:36.929823 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.929788    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"]
Jul 24 19:28:36.933504 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.933483    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"
Jul 24 19:28:36.936029 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.935992    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-5ce07c\""
Jul 24 19:28:36.936161 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.936119    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-5ce07c-dockercfg-9nr5k\""
Jul 24 19:28:36.944048 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:36.944023    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"]
Jul 24 19:28:37.087241 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:37.087201    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4503a25-3dac-40ac-9717-627c86ad4d6f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj\" (UID: \"f4503a25-3dac-40ac-9717-627c86ad4d6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"
Jul 24 19:28:37.087470 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:37.087254    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbss9\" (UniqueName: \"kubernetes.io/projected/f4503a25-3dac-40ac-9717-627c86ad4d6f-kube-api-access-vbss9\") pod \"payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj\" (UID: \"f4503a25-3dac-40ac-9717-627c86ad4d6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"
Jul 24 19:28:37.119201 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:37.119167    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:28:37.188574 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:37.188533    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4503a25-3dac-40ac-9717-627c86ad4d6f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj\" (UID: \"f4503a25-3dac-40ac-9717-627c86ad4d6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"
Jul 24 19:28:37.188727 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:37.188636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbss9\" (UniqueName: \"kubernetes.io/projected/f4503a25-3dac-40ac-9717-627c86ad4d6f-kube-api-access-vbss9\") pod \"payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj\" (UID: \"f4503a25-3dac-40ac-9717-627c86ad4d6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"
Jul 24 19:28:37.189213 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:37.189194    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4503a25-3dac-40ac-9717-627c86ad4d6f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj\" (UID: \"f4503a25-3dac-40ac-9717-627c86ad4d6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"
Jul 24 19:28:37.197902 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:37.197873    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbss9\" (UniqueName: \"kubernetes.io/projected/f4503a25-3dac-40ac-9717-627c86ad4d6f-kube-api-access-vbss9\") pod \"payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj\" (UID: \"f4503a25-3dac-40ac-9717-627c86ad4d6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"
Jul 24 19:28:37.247050 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:37.247014    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"
Jul 24 19:28:37.256760 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:37.256739    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"]
Jul 24 19:28:37.259321 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:28:37.259292    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4124d1fa_edba_4079_a170_b30c203f545e.slice/crio-b751a6903ba29288515f08914bc881bbba35856ab5e112daed2d03903105509b WatchSource:0}: Error finding container b751a6903ba29288515f08914bc881bbba35856ab5e112daed2d03903105509b: Status 404 returned error can't find the container with id b751a6903ba29288515f08914bc881bbba35856ab5e112daed2d03903105509b
Jul 24 19:28:37.388480 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:37.388456    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"]
Jul 24 19:28:37.389632 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:28:37.389608    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4503a25_3dac_40ac_9717_627c86ad4d6f.slice/crio-27b1e9a38ff4eac6f5d4f1b6caf8f31ef6d9398de40fe3202f22c5ec4a65d87c WatchSource:0}: Error finding container 27b1e9a38ff4eac6f5d4f1b6caf8f31ef6d9398de40fe3202f22c5ec4a65d87c: Status 404 returned error can't find the container with id 27b1e9a38ff4eac6f5d4f1b6caf8f31ef6d9398de40fe3202f22c5ec4a65d87c
Jul 24 19:28:38.213012 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:38.212969    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj" event={"ID":"f4503a25-3dac-40ac-9717-627c86ad4d6f","Type":"ContainerStarted","Data":"4a98058aaed4bf200ee9be2a7caa5feb729ff674f6e252fe519459d251478508"}
Jul 24 19:28:38.213225 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:38.213034    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj" event={"ID":"f4503a25-3dac-40ac-9717-627c86ad4d6f","Type":"ContainerStarted","Data":"27b1e9a38ff4eac6f5d4f1b6caf8f31ef6d9398de40fe3202f22c5ec4a65d87c"}
Jul 24 19:28:38.213225 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:38.213064    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"
Jul 24 19:28:38.214476 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:38.214455    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n" event={"ID":"4124d1fa-edba-4079-a170-b30c203f545e","Type":"ContainerStarted","Data":"6a078282beef6e3e1dc9d7bd1a02c8e9aa997bbef486972d7c4bb5b745925a88"}
Jul 24 19:28:38.214592 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:38.214494    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n" event={"ID":"4124d1fa-edba-4079-a170-b30c203f545e","Type":"ContainerStarted","Data":"b751a6903ba29288515f08914bc881bbba35856ab5e112daed2d03903105509b"}
Jul 24 19:28:38.214592 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:38.214528    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:28:38.235824 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:38.235778    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj" podStartSLOduration=2.235765502 podStartE2EDuration="2.235765502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:28:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:28:38.233838541 +0000 UTC m=+2656.624790018" watchObservedRunningTime="2026-07-24 19:28:38.235765502 +0000 UTC m=+2656.626717030"
Jul 24 19:28:38.253322 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:38.253269    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n" podStartSLOduration=1.873465481 podStartE2EDuration="2.253255835s" totalImagesPullingTime="379.790354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:28:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:28:38.250840995 +0000 UTC m=+2656.641792474" watchObservedRunningTime="2026-07-24 19:28:38.253255835 +0000 UTC m=+2656.644207313"
Jul 24 19:28:39.355418 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:39.355371    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:28:44.225796 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:44.225762    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:28:49.222830 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:49.222793    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"
Jul 24 19:28:57.913456 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:28:57.913385    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj_f4503a25-3dac-40ac-9717-627c86ad4d6f/payload-processing/0.log"
Jul 24 19:29:10.051754 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:29:10.051714    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:29:22.299342 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:29:22.299267    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 19:29:22.307883 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:29:22.307861    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-889p5_e5a77bc5-eeda-4e10-872e-190ad4ffeff5/ovn-acl-logging/0.log"
Jul 24 19:29:25.653020 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:29:25.652983    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:29:36.550056 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:29:36.550019    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52_2bd845bf-485c-4ce3-8d89-79b7a99dcf71/payload-processing/0.log"
Jul 24 19:29:36.777907 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:29:36.777879    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj_f4503a25-3dac-40ac-9717-627c86ad4d6f/payload-processing/0.log"
Jul 24 19:29:47.251980 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:29:47.251940    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:30:13.156946 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:13.156911    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:30:37.057982 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:37.057952    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:30:37.725252 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:37.725217    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"]
Jul 24 19:30:37.791088 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:37.791016    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"]
Jul 24 19:30:37.791585 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:37.791529    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52" podUID="2bd845bf-485c-4ce3-8d89-79b7a99dcf71" containerName="payload-processing" containerID="cri-o://0bac2c475bf5e229d080251393bf7483c09ea28f07d1e2345f13a58de0db4839" gracePeriod=30
Jul 24 19:30:37.799093 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:37.799068    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"]
Jul 24 19:30:37.799843 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:37.799371    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t" podUID="b0d0bdef-5d75-4a72-8132-c2b076f17a85" containerName="payload-pre-processing" containerID="cri-o://4d912c7618181565e523c149f50ce463798267b9abab4831874d6a8d2fa7f832" gracePeriod=30
Jul 24 19:30:37.874501 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:37.874469    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc" podUID="fa0f22d5-9afe-4d04-bbe8-b63075f9bd85" containerName="maas-api" containerID="cri-o://1a4175910ccd6e38ab7238a5b85aeffe519200301019060d5bf68c91913b1bad" gracePeriod=30
Jul 24 19:30:38.142946 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.142926    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"
Jul 24 19:30:38.158438 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.158417    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"
Jul 24 19:30:38.160142 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.160119    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0d0bdef-5d75-4a72-8132-c2b076f17a85-plugins-config-volume\") pod \"b0d0bdef-5d75-4a72-8132-c2b076f17a85\" (UID: \"b0d0bdef-5d75-4a72-8132-c2b076f17a85\") "
Jul 24 19:30:38.160264 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.160185    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8d67k\" (UniqueName: \"kubernetes.io/projected/b0d0bdef-5d75-4a72-8132-c2b076f17a85-kube-api-access-8d67k\") pod \"b0d0bdef-5d75-4a72-8132-c2b076f17a85\" (UID: \"b0d0bdef-5d75-4a72-8132-c2b076f17a85\") "
Jul 24 19:30:38.160778 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.160745    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0d0bdef-5d75-4a72-8132-c2b076f17a85-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b0d0bdef-5d75-4a72-8132-c2b076f17a85" (UID: "b0d0bdef-5d75-4a72-8132-c2b076f17a85"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:30:38.163690 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.163661    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0d0bdef-5d75-4a72-8132-c2b076f17a85-kube-api-access-8d67k" (OuterVolumeSpecName: "kube-api-access-8d67k") pod "b0d0bdef-5d75-4a72-8132-c2b076f17a85" (UID: "b0d0bdef-5d75-4a72-8132-c2b076f17a85"). InnerVolumeSpecName "kube-api-access-8d67k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:30:38.168299 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.168278    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:30:38.261509 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.261426    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tczk\" (UniqueName: \"kubernetes.io/projected/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-kube-api-access-4tczk\") pod \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\" (UID: \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\") "
Jul 24 19:30:38.261509 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.261464    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hbf7\" (UniqueName: \"kubernetes.io/projected/2bd845bf-485c-4ce3-8d89-79b7a99dcf71-kube-api-access-4hbf7\") pod \"2bd845bf-485c-4ce3-8d89-79b7a99dcf71\" (UID: \"2bd845bf-485c-4ce3-8d89-79b7a99dcf71\") "
Jul 24 19:30:38.261509 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.261492    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-maas-api-tls\") pod \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\" (UID: \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\") "
Jul 24 19:30:38.261783 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.261607    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-openshift-service-ca\") pod \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\" (UID: \"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85\") "
Jul 24 19:30:38.261783 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.261649    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bd845bf-485c-4ce3-8d89-79b7a99dcf71-plugins-config-volume\") pod \"2bd845bf-485c-4ce3-8d89-79b7a99dcf71\" (UID: \"2bd845bf-485c-4ce3-8d89-79b7a99dcf71\") "
Jul 24 19:30:38.261970 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.261951    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0d0bdef-5d75-4a72-8132-c2b076f17a85-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:30:38.262089 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.261951    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fa0f22d5-9afe-4d04-bbe8-b63075f9bd85" (UID: "fa0f22d5-9afe-4d04-bbe8-b63075f9bd85"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:30:38.262089 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.261971    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d67k\" (UniqueName: \"kubernetes.io/projected/b0d0bdef-5d75-4a72-8132-c2b076f17a85-kube-api-access-8d67k\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:30:38.262089 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.262070    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bd845bf-485c-4ce3-8d89-79b7a99dcf71-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2bd845bf-485c-4ce3-8d89-79b7a99dcf71" (UID: "2bd845bf-485c-4ce3-8d89-79b7a99dcf71"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:30:38.263807 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.263786    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-kube-api-access-4tczk" (OuterVolumeSpecName: "kube-api-access-4tczk") pod "fa0f22d5-9afe-4d04-bbe8-b63075f9bd85" (UID: "fa0f22d5-9afe-4d04-bbe8-b63075f9bd85"). InnerVolumeSpecName "kube-api-access-4tczk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:30:38.264028 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.264011    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fa0f22d5-9afe-4d04-bbe8-b63075f9bd85" (UID: "fa0f22d5-9afe-4d04-bbe8-b63075f9bd85"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:30:38.264098 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.264084    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bd845bf-485c-4ce3-8d89-79b7a99dcf71-kube-api-access-4hbf7" (OuterVolumeSpecName: "kube-api-access-4hbf7") pod "2bd845bf-485c-4ce3-8d89-79b7a99dcf71" (UID: "2bd845bf-485c-4ce3-8d89-79b7a99dcf71"). InnerVolumeSpecName "kube-api-access-4hbf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:30:38.362679 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.362640    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tczk\" (UniqueName: \"kubernetes.io/projected/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-kube-api-access-4tczk\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:30:38.362679 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.362676    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hbf7\" (UniqueName: \"kubernetes.io/projected/2bd845bf-485c-4ce3-8d89-79b7a99dcf71-kube-api-access-4hbf7\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:30:38.362679 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.362686    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:30:38.362884 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.362696    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:30:38.362884 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.362707    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bd845bf-485c-4ce3-8d89-79b7a99dcf71-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:30:38.881114 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.881079    2563 generic.go:356] "Generic (PLEG): container finished" podID="fa0f22d5-9afe-4d04-bbe8-b63075f9bd85" containerID="1a4175910ccd6e38ab7238a5b85aeffe519200301019060d5bf68c91913b1bad" exitCode=0
Jul 24 19:30:38.881288 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.881131    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"
Jul 24 19:30:38.881288 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.881169    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc" event={"ID":"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85","Type":"ContainerDied","Data":"1a4175910ccd6e38ab7238a5b85aeffe519200301019060d5bf68c91913b1bad"}
Jul 24 19:30:38.881288 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.881224    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc" event={"ID":"fa0f22d5-9afe-4d04-bbe8-b63075f9bd85","Type":"ContainerDied","Data":"cfeb605e97c654c261cd8a4932f5a4c09e387a6f05e9b812355364c5e588b595"}
Jul 24 19:30:38.881288 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.881246    2563 scope.go:117] "RemoveContainer" containerID="1a4175910ccd6e38ab7238a5b85aeffe519200301019060d5bf68c91913b1bad"
Jul 24 19:30:38.882777 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.882757    2563 generic.go:356] "Generic (PLEG): container finished" podID="2bd845bf-485c-4ce3-8d89-79b7a99dcf71" containerID="0bac2c475bf5e229d080251393bf7483c09ea28f07d1e2345f13a58de0db4839" exitCode=0
Jul 24 19:30:38.882880 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.882869    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52" event={"ID":"2bd845bf-485c-4ce3-8d89-79b7a99dcf71","Type":"ContainerDied","Data":"0bac2c475bf5e229d080251393bf7483c09ea28f07d1e2345f13a58de0db4839"}
Jul 24 19:30:38.882926 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.882905    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52" event={"ID":"2bd845bf-485c-4ce3-8d89-79b7a99dcf71","Type":"ContainerDied","Data":"2621de9fb9dcc4e2d3a8afa4887ae1ded5be0970b5edaae94d58f8e3c68d8cb3"}
Jul 24 19:30:38.882974 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.882947    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"
Jul 24 19:30:38.884965 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.884939    2563 generic.go:356] "Generic (PLEG): container finished" podID="b0d0bdef-5d75-4a72-8132-c2b076f17a85" containerID="4d912c7618181565e523c149f50ce463798267b9abab4831874d6a8d2fa7f832" exitCode=0
Jul 24 19:30:38.885111 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.884984    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t" event={"ID":"b0d0bdef-5d75-4a72-8132-c2b076f17a85","Type":"ContainerDied","Data":"4d912c7618181565e523c149f50ce463798267b9abab4831874d6a8d2fa7f832"}
Jul 24 19:30:38.885111 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.885018    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"
Jul 24 19:30:38.885111 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.885021    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t" event={"ID":"b0d0bdef-5d75-4a72-8132-c2b076f17a85","Type":"ContainerDied","Data":"020fd39ff7f9a37a9a4582356b9343741b04bcbb994bf0d722501a45dacfbcc8"}
Jul 24 19:30:38.892556 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.892538    2563 scope.go:117] "RemoveContainer" containerID="1a4175910ccd6e38ab7238a5b85aeffe519200301019060d5bf68c91913b1bad"
Jul 24 19:30:38.892948 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:30:38.892929    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a4175910ccd6e38ab7238a5b85aeffe519200301019060d5bf68c91913b1bad\": container with ID starting with 1a4175910ccd6e38ab7238a5b85aeffe519200301019060d5bf68c91913b1bad not found: ID does not exist" containerID="1a4175910ccd6e38ab7238a5b85aeffe519200301019060d5bf68c91913b1bad"
Jul 24 19:30:38.893016 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.892956    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a4175910ccd6e38ab7238a5b85aeffe519200301019060d5bf68c91913b1bad"} err="failed to get container status \"1a4175910ccd6e38ab7238a5b85aeffe519200301019060d5bf68c91913b1bad\": rpc error: code = NotFound desc = could not find container \"1a4175910ccd6e38ab7238a5b85aeffe519200301019060d5bf68c91913b1bad\": container with ID starting with 1a4175910ccd6e38ab7238a5b85aeffe519200301019060d5bf68c91913b1bad not found: ID does not exist"
Jul 24 19:30:38.893016 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.892977    2563 scope.go:117] "RemoveContainer" containerID="0bac2c475bf5e229d080251393bf7483c09ea28f07d1e2345f13a58de0db4839"
Jul 24 19:30:38.900993 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.900976    2563 scope.go:117] "RemoveContainer" containerID="0bac2c475bf5e229d080251393bf7483c09ea28f07d1e2345f13a58de0db4839"
Jul 24 19:30:38.901286 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:30:38.901263    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bac2c475bf5e229d080251393bf7483c09ea28f07d1e2345f13a58de0db4839\": container with ID starting with 0bac2c475bf5e229d080251393bf7483c09ea28f07d1e2345f13a58de0db4839 not found: ID does not exist" containerID="0bac2c475bf5e229d080251393bf7483c09ea28f07d1e2345f13a58de0db4839"
Jul 24 19:30:38.901373 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.901294    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bac2c475bf5e229d080251393bf7483c09ea28f07d1e2345f13a58de0db4839"} err="failed to get container status \"0bac2c475bf5e229d080251393bf7483c09ea28f07d1e2345f13a58de0db4839\": rpc error: code = NotFound desc = could not find container \"0bac2c475bf5e229d080251393bf7483c09ea28f07d1e2345f13a58de0db4839\": container with ID starting with 0bac2c475bf5e229d080251393bf7483c09ea28f07d1e2345f13a58de0db4839 not found: ID does not exist"
Jul 24 19:30:38.901373 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.901318    2563 scope.go:117] "RemoveContainer" containerID="4d912c7618181565e523c149f50ce463798267b9abab4831874d6a8d2fa7f832"
Jul 24 19:30:38.904073 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.904050    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"]
Jul 24 19:30:38.908255 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.908234    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-87cd66-5fb455848d-4zc2t"]
Jul 24 19:30:38.910344 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.910320    2563 scope.go:117] "RemoveContainer" containerID="4d912c7618181565e523c149f50ce463798267b9abab4831874d6a8d2fa7f832"
Jul 24 19:30:38.910660 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:30:38.910635    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d912c7618181565e523c149f50ce463798267b9abab4831874d6a8d2fa7f832\": container with ID starting with 4d912c7618181565e523c149f50ce463798267b9abab4831874d6a8d2fa7f832 not found: ID does not exist" containerID="4d912c7618181565e523c149f50ce463798267b9abab4831874d6a8d2fa7f832"
Jul 24 19:30:38.910729 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.910669    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d912c7618181565e523c149f50ce463798267b9abab4831874d6a8d2fa7f832"} err="failed to get container status \"4d912c7618181565e523c149f50ce463798267b9abab4831874d6a8d2fa7f832\": rpc error: code = NotFound desc = could not find container \"4d912c7618181565e523c149f50ce463798267b9abab4831874d6a8d2fa7f832\": container with ID starting with 4d912c7618181565e523c149f50ce463798267b9abab4831874d6a8d2fa7f832 not found: ID does not exist"
Jul 24 19:30:38.917712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.917687    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"]
Jul 24 19:30:38.921301 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.921280    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-87cd66-574dbf9d98-d2w52"]
Jul 24 19:30:38.930644 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.930614    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"]
Jul 24 19:30:38.934600 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:38.934580    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-87cd66-77bfbdd4d8-psxwc"]
Jul 24 19:30:40.210312 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:40.210272    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bd845bf-485c-4ce3-8d89-79b7a99dcf71" path="/var/lib/kubelet/pods/2bd845bf-485c-4ce3-8d89-79b7a99dcf71/volumes"
Jul 24 19:30:40.210728 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:40.210712    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0d0bdef-5d75-4a72-8132-c2b076f17a85" path="/var/lib/kubelet/pods/b0d0bdef-5d75-4a72-8132-c2b076f17a85/volumes"
Jul 24 19:30:40.211053 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:40.211041    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa0f22d5-9afe-4d04-bbe8-b63075f9bd85" path="/var/lib/kubelet/pods/fa0f22d5-9afe-4d04-bbe8-b63075f9bd85/volumes"
Jul 24 19:30:47.453166 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:30:47.453125    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:31:41.151783 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.151744    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"]
Jul 24 19:31:41.152355 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.151997    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n" podUID="4124d1fa-edba-4079-a170-b30c203f545e" containerName="maas-api" containerID="cri-o://6a078282beef6e3e1dc9d7bd1a02c8e9aa997bbef486972d7c4bb5b745925a88" gracePeriod=30
Jul 24 19:31:41.216932 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.216902    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"]
Jul 24 19:31:41.217135 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.217110    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj" podUID="f4503a25-3dac-40ac-9717-627c86ad4d6f" containerName="payload-processing" containerID="cri-o://4a98058aaed4bf200ee9be2a7caa5feb729ff674f6e252fe519459d251478508" gracePeriod=30
Jul 24 19:31:41.463132 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.463102    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:31:41.496735 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.496712    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"
Jul 24 19:31:41.502514 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.502490    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkp7w\" (UniqueName: \"kubernetes.io/projected/4124d1fa-edba-4079-a170-b30c203f545e-kube-api-access-qkp7w\") pod \"4124d1fa-edba-4079-a170-b30c203f545e\" (UID: \"4124d1fa-edba-4079-a170-b30c203f545e\") "
Jul 24 19:31:41.502626 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.502554    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4124d1fa-edba-4079-a170-b30c203f545e-openshift-service-ca\") pod \"4124d1fa-edba-4079-a170-b30c203f545e\" (UID: \"4124d1fa-edba-4079-a170-b30c203f545e\") "
Jul 24 19:31:41.502626 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.502582    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4124d1fa-edba-4079-a170-b30c203f545e-maas-api-tls\") pod \"4124d1fa-edba-4079-a170-b30c203f545e\" (UID: \"4124d1fa-edba-4079-a170-b30c203f545e\") "
Jul 24 19:31:41.502904 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.502882    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4124d1fa-edba-4079-a170-b30c203f545e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4124d1fa-edba-4079-a170-b30c203f545e" (UID: "4124d1fa-edba-4079-a170-b30c203f545e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:31:41.504692 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.504671    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4124d1fa-edba-4079-a170-b30c203f545e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4124d1fa-edba-4079-a170-b30c203f545e" (UID: "4124d1fa-edba-4079-a170-b30c203f545e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:31:41.504762 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.504685    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4124d1fa-edba-4079-a170-b30c203f545e-kube-api-access-qkp7w" (OuterVolumeSpecName: "kube-api-access-qkp7w") pod "4124d1fa-edba-4079-a170-b30c203f545e" (UID: "4124d1fa-edba-4079-a170-b30c203f545e"). InnerVolumeSpecName "kube-api-access-qkp7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:31:41.603714 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.603672    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4503a25-3dac-40ac-9717-627c86ad4d6f-plugins-config-volume\") pod \"f4503a25-3dac-40ac-9717-627c86ad4d6f\" (UID: \"f4503a25-3dac-40ac-9717-627c86ad4d6f\") "
Jul 24 19:31:41.603714 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.603715    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbss9\" (UniqueName: \"kubernetes.io/projected/f4503a25-3dac-40ac-9717-627c86ad4d6f-kube-api-access-vbss9\") pod \"f4503a25-3dac-40ac-9717-627c86ad4d6f\" (UID: \"f4503a25-3dac-40ac-9717-627c86ad4d6f\") "
Jul 24 19:31:41.603980 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.603833    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkp7w\" (UniqueName: \"kubernetes.io/projected/4124d1fa-edba-4079-a170-b30c203f545e-kube-api-access-qkp7w\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:31:41.603980 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.603846    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4124d1fa-edba-4079-a170-b30c203f545e-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:31:41.603980 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.603855    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4124d1fa-edba-4079-a170-b30c203f545e-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:31:41.604142 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.604094    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4503a25-3dac-40ac-9717-627c86ad4d6f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f4503a25-3dac-40ac-9717-627c86ad4d6f" (UID: "f4503a25-3dac-40ac-9717-627c86ad4d6f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:31:41.605955 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.605914    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4503a25-3dac-40ac-9717-627c86ad4d6f-kube-api-access-vbss9" (OuterVolumeSpecName: "kube-api-access-vbss9") pod "f4503a25-3dac-40ac-9717-627c86ad4d6f" (UID: "f4503a25-3dac-40ac-9717-627c86ad4d6f"). InnerVolumeSpecName "kube-api-access-vbss9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:31:41.704458 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.704352    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4503a25-3dac-40ac-9717-627c86ad4d6f-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:31:41.704458 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:41.704425    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbss9\" (UniqueName: \"kubernetes.io/projected/f4503a25-3dac-40ac-9717-627c86ad4d6f-kube-api-access-vbss9\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:31:42.221598 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.221573    2563 generic.go:356] "Generic (PLEG): container finished" podID="f4503a25-3dac-40ac-9717-627c86ad4d6f" containerID="4a98058aaed4bf200ee9be2a7caa5feb729ff674f6e252fe519459d251478508" exitCode=0
Jul 24 19:31:42.221939 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.221615    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"
Jul 24 19:31:42.221939 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.221655    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj" event={"ID":"f4503a25-3dac-40ac-9717-627c86ad4d6f","Type":"ContainerDied","Data":"4a98058aaed4bf200ee9be2a7caa5feb729ff674f6e252fe519459d251478508"}
Jul 24 19:31:42.221939 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.221709    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj" event={"ID":"f4503a25-3dac-40ac-9717-627c86ad4d6f","Type":"ContainerDied","Data":"27b1e9a38ff4eac6f5d4f1b6caf8f31ef6d9398de40fe3202f22c5ec4a65d87c"}
Jul 24 19:31:42.221939 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.221739    2563 scope.go:117] "RemoveContainer" containerID="4a98058aaed4bf200ee9be2a7caa5feb729ff674f6e252fe519459d251478508"
Jul 24 19:31:42.223069 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.223051    2563 generic.go:356] "Generic (PLEG): container finished" podID="4124d1fa-edba-4079-a170-b30c203f545e" containerID="6a078282beef6e3e1dc9d7bd1a02c8e9aa997bbef486972d7c4bb5b745925a88" exitCode=0
Jul 24 19:31:42.223136 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.223089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n" event={"ID":"4124d1fa-edba-4079-a170-b30c203f545e","Type":"ContainerDied","Data":"6a078282beef6e3e1dc9d7bd1a02c8e9aa997bbef486972d7c4bb5b745925a88"}
Jul 24 19:31:42.223136 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.223108    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"
Jul 24 19:31:42.223136 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.223118    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n" event={"ID":"4124d1fa-edba-4079-a170-b30c203f545e","Type":"ContainerDied","Data":"b751a6903ba29288515f08914bc881bbba35856ab5e112daed2d03903105509b"}
Jul 24 19:31:42.230803 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.230771    2563 scope.go:117] "RemoveContainer" containerID="4a98058aaed4bf200ee9be2a7caa5feb729ff674f6e252fe519459d251478508"
Jul 24 19:31:42.231058 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:31:42.231040    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a98058aaed4bf200ee9be2a7caa5feb729ff674f6e252fe519459d251478508\": container with ID starting with 4a98058aaed4bf200ee9be2a7caa5feb729ff674f6e252fe519459d251478508 not found: ID does not exist" containerID="4a98058aaed4bf200ee9be2a7caa5feb729ff674f6e252fe519459d251478508"
Jul 24 19:31:42.231107 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.231066    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a98058aaed4bf200ee9be2a7caa5feb729ff674f6e252fe519459d251478508"} err="failed to get container status \"4a98058aaed4bf200ee9be2a7caa5feb729ff674f6e252fe519459d251478508\": rpc error: code = NotFound desc = could not find container \"4a98058aaed4bf200ee9be2a7caa5feb729ff674f6e252fe519459d251478508\": container with ID starting with 4a98058aaed4bf200ee9be2a7caa5feb729ff674f6e252fe519459d251478508 not found: ID does not exist"
Jul 24 19:31:42.231107 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.231083    2563 scope.go:117] "RemoveContainer" containerID="6a078282beef6e3e1dc9d7bd1a02c8e9aa997bbef486972d7c4bb5b745925a88"
Jul 24 19:31:42.238749 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.238730    2563 scope.go:117] "RemoveContainer" containerID="6a078282beef6e3e1dc9d7bd1a02c8e9aa997bbef486972d7c4bb5b745925a88"
Jul 24 19:31:42.238820 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.238741    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"]
Jul 24 19:31:42.239011 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:31:42.238994    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a078282beef6e3e1dc9d7bd1a02c8e9aa997bbef486972d7c4bb5b745925a88\": container with ID starting with 6a078282beef6e3e1dc9d7bd1a02c8e9aa997bbef486972d7c4bb5b745925a88 not found: ID does not exist" containerID="6a078282beef6e3e1dc9d7bd1a02c8e9aa997bbef486972d7c4bb5b745925a88"
Jul 24 19:31:42.239055 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.239017    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a078282beef6e3e1dc9d7bd1a02c8e9aa997bbef486972d7c4bb5b745925a88"} err="failed to get container status \"6a078282beef6e3e1dc9d7bd1a02c8e9aa997bbef486972d7c4bb5b745925a88\": rpc error: code = NotFound desc = could not find container \"6a078282beef6e3e1dc9d7bd1a02c8e9aa997bbef486972d7c4bb5b745925a88\": container with ID starting with 6a078282beef6e3e1dc9d7bd1a02c8e9aa997bbef486972d7c4bb5b745925a88 not found: ID does not exist"
Jul 24 19:31:42.242699 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.242680    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5ce07c-78fbd6c85d-cwwhj"]
Jul 24 19:31:42.251783 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.251764    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"]
Jul 24 19:31:42.254285 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:42.254267    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5ce07c-fc466cbdb-rx86n"]
Jul 24 19:31:44.203660 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:44.203627    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4124d1fa-edba-4079-a170-b30c203f545e" path="/var/lib/kubelet/pods/4124d1fa-edba-4079-a170-b30c203f545e/volumes"
Jul 24 19:31:44.204066 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:44.203974    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4503a25-3dac-40ac-9717-627c86ad4d6f" path="/var/lib/kubelet/pods/f4503a25-3dac-40ac-9717-627c86ad4d6f/volumes"
Jul 24 19:31:44.345534 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:31:44.345500    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:32:03.016096 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.016056    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"]
Jul 24 19:32:03.017030 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.016980    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv" podUID="842bf3a4-5e60-4045-a932-6f569740ee01" containerName="maas-api" containerID="cri-o://4a95632abc1277564b3f7b10f59bf4e5b8b2ddde7227c0bdbbe19efd5a013be1" gracePeriod=30
Jul 24 19:32:03.077851 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.077815    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"]
Jul 24 19:32:03.078077 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.078045    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk" podUID="78a8e8a6-da1d-4685-b288-0092732f0458" containerName="payload-processing" containerID="cri-o://e769038001149990f09d4da9649604e6744542152586f5cb2ca3e323b86ae515" gracePeriod=30
Jul 24 19:32:03.332824 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.332796    2563 generic.go:356] "Generic (PLEG): container finished" podID="78a8e8a6-da1d-4685-b288-0092732f0458" containerID="e769038001149990f09d4da9649604e6744542152586f5cb2ca3e323b86ae515" exitCode=0
Jul 24 19:32:03.332960 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.332865    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk" event={"ID":"78a8e8a6-da1d-4685-b288-0092732f0458","Type":"ContainerDied","Data":"e769038001149990f09d4da9649604e6744542152586f5cb2ca3e323b86ae515"}
Jul 24 19:32:03.334386 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.334358    2563 generic.go:356] "Generic (PLEG): container finished" podID="842bf3a4-5e60-4045-a932-6f569740ee01" containerID="4a95632abc1277564b3f7b10f59bf4e5b8b2ddde7227c0bdbbe19efd5a013be1" exitCode=0
Jul 24 19:32:03.334543 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.334450    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv" event={"ID":"842bf3a4-5e60-4045-a932-6f569740ee01","Type":"ContainerDied","Data":"4a95632abc1277564b3f7b10f59bf4e5b8b2ddde7227c0bdbbe19efd5a013be1"}
Jul 24 19:32:03.353928 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.353904    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:32:03.356573 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.356548    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"
Jul 24 19:32:03.487307 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.487267    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/842bf3a4-5e60-4045-a932-6f569740ee01-maas-api-tls\") pod \"842bf3a4-5e60-4045-a932-6f569740ee01\" (UID: \"842bf3a4-5e60-4045-a932-6f569740ee01\") "
Jul 24 19:32:03.487528 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.487373    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78a8e8a6-da1d-4685-b288-0092732f0458-plugins-config-volume\") pod \"78a8e8a6-da1d-4685-b288-0092732f0458\" (UID: \"78a8e8a6-da1d-4685-b288-0092732f0458\") "
Jul 24 19:32:03.487528 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.487435    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdpzv\" (UniqueName: \"kubernetes.io/projected/78a8e8a6-da1d-4685-b288-0092732f0458-kube-api-access-rdpzv\") pod \"78a8e8a6-da1d-4685-b288-0092732f0458\" (UID: \"78a8e8a6-da1d-4685-b288-0092732f0458\") "
Jul 24 19:32:03.487528 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.487464    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtbdm\" (UniqueName: \"kubernetes.io/projected/842bf3a4-5e60-4045-a932-6f569740ee01-kube-api-access-qtbdm\") pod \"842bf3a4-5e60-4045-a932-6f569740ee01\" (UID: \"842bf3a4-5e60-4045-a932-6f569740ee01\") "
Jul 24 19:32:03.487707 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.487595    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/842bf3a4-5e60-4045-a932-6f569740ee01-openshift-service-ca\") pod \"842bf3a4-5e60-4045-a932-6f569740ee01\" (UID: \"842bf3a4-5e60-4045-a932-6f569740ee01\") "
Jul 24 19:32:03.487815 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.487785    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78a8e8a6-da1d-4685-b288-0092732f0458-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "78a8e8a6-da1d-4685-b288-0092732f0458" (UID: "78a8e8a6-da1d-4685-b288-0092732f0458"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:32:03.487953 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.487924    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/842bf3a4-5e60-4045-a932-6f569740ee01-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "842bf3a4-5e60-4045-a932-6f569740ee01" (UID: "842bf3a4-5e60-4045-a932-6f569740ee01"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:32:03.488019 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.487974    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78a8e8a6-da1d-4685-b288-0092732f0458-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:03.489706 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.489684    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/842bf3a4-5e60-4045-a932-6f569740ee01-kube-api-access-qtbdm" (OuterVolumeSpecName: "kube-api-access-qtbdm") pod "842bf3a4-5e60-4045-a932-6f569740ee01" (UID: "842bf3a4-5e60-4045-a932-6f569740ee01"). InnerVolumeSpecName "kube-api-access-qtbdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:32:03.490109 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.490089    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78a8e8a6-da1d-4685-b288-0092732f0458-kube-api-access-rdpzv" (OuterVolumeSpecName: "kube-api-access-rdpzv") pod "78a8e8a6-da1d-4685-b288-0092732f0458" (UID: "78a8e8a6-da1d-4685-b288-0092732f0458"). InnerVolumeSpecName "kube-api-access-rdpzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:32:03.490109 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.490093    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/842bf3a4-5e60-4045-a932-6f569740ee01-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "842bf3a4-5e60-4045-a932-6f569740ee01" (UID: "842bf3a4-5e60-4045-a932-6f569740ee01"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:32:03.588802 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.588713    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdpzv\" (UniqueName: \"kubernetes.io/projected/78a8e8a6-da1d-4685-b288-0092732f0458-kube-api-access-rdpzv\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:03.588802 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.588742    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtbdm\" (UniqueName: \"kubernetes.io/projected/842bf3a4-5e60-4045-a932-6f569740ee01-kube-api-access-qtbdm\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:03.588802 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.588753    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/842bf3a4-5e60-4045-a932-6f569740ee01-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:03.588802 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:03.588762    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/842bf3a4-5e60-4045-a932-6f569740ee01-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:04.340703 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:04.340672    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"
Jul 24 19:32:04.340703 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:04.340689    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk" event={"ID":"78a8e8a6-da1d-4685-b288-0092732f0458","Type":"ContainerDied","Data":"937fdb98b1fe54b81e6b1bdfd424b6a49570a3b3df6eb3d0377b5e5e19fbd9a7"}
Jul 24 19:32:04.341228 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:04.340761    2563 scope.go:117] "RemoveContainer" containerID="e769038001149990f09d4da9649604e6744542152586f5cb2ca3e323b86ae515"
Jul 24 19:32:04.342171 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:04.342147    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv" event={"ID":"842bf3a4-5e60-4045-a932-6f569740ee01","Type":"ContainerDied","Data":"26cd046efae0d01b2162b3341ac1ed7e80efce3b28d79b349b72f428d1eac9af"}
Jul 24 19:32:04.342258 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:04.342156    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"
Jul 24 19:32:04.350031 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:04.350010    2563 scope.go:117] "RemoveContainer" containerID="4a95632abc1277564b3f7b10f59bf4e5b8b2ddde7227c0bdbbe19efd5a013be1"
Jul 24 19:32:04.362936 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:04.362914    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"]
Jul 24 19:32:04.368131 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:04.368111    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e05f4d-84897ffb9d-h8gcv"]
Jul 24 19:32:04.377339 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:04.377315    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"]
Jul 24 19:32:04.384322 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:04.384297    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-e05f4d-55445694c4-8h9tk"]
Jul 24 19:32:06.169988 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:06.169950    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:32:06.203894 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:06.203865    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78a8e8a6-da1d-4685-b288-0092732f0458" path="/var/lib/kubelet/pods/78a8e8a6-da1d-4685-b288-0092732f0458/volumes"
Jul 24 19:32:06.204230 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:06.204218    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="842bf3a4-5e60-4045-a932-6f569740ee01" path="/var/lib/kubelet/pods/842bf3a4-5e60-4045-a932-6f569740ee01/volumes"
Jul 24 19:32:14.038630 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:14.038544    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"]
Jul 24 19:32:14.039042 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:14.038810    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw" podUID="91816a65-55c7-452d-ba71-26266c4908d4" containerName="istio-proxy" containerID="cri-o://de2929d0cc8636b83d341a6b883ec2f68536f1dc17e03fb01729ec0ccea9390a" gracePeriod=30
Jul 24 19:32:15.036709 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:15.036676    2563 patch_prober.go:28] interesting pod/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 19:32:15.036884 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:15.036732    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw" podUID="91816a65-55c7-452d-ba71-26266c4908d4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 19:32:19.284705 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.284676    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:32:19.319063 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.319037    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njmp9\" (UniqueName: \"kubernetes.io/projected/91816a65-55c7-452d-ba71-26266c4908d4-kube-api-access-njmp9\") pod \"91816a65-55c7-452d-ba71-26266c4908d4\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") "
Jul 24 19:32:19.319225 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.319098    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-istio-envoy\") pod \"91816a65-55c7-452d-ba71-26266c4908d4\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") "
Jul 24 19:32:19.319225 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.319153    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-workload-socket\") pod \"91816a65-55c7-452d-ba71-26266c4908d4\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") "
Jul 24 19:32:19.319225 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.319177    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-istio-data\") pod \"91816a65-55c7-452d-ba71-26266c4908d4\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") "
Jul 24 19:32:19.319225 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.319198    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91816a65-55c7-452d-ba71-26266c4908d4-istio-token\") pod \"91816a65-55c7-452d-ba71-26266c4908d4\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") "
Jul 24 19:32:19.319474 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.319259    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91816a65-55c7-452d-ba71-26266c4908d4-istio-podinfo\") pod \"91816a65-55c7-452d-ba71-26266c4908d4\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") "
Jul 24 19:32:19.319474 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.319296    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-credential-socket\") pod \"91816a65-55c7-452d-ba71-26266c4908d4\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") "
Jul 24 19:32:19.319474 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.319326    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-workload-certs\") pod \"91816a65-55c7-452d-ba71-26266c4908d4\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") "
Jul 24 19:32:19.319474 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.319355    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91816a65-55c7-452d-ba71-26266c4908d4-istiod-ca-cert\") pod \"91816a65-55c7-452d-ba71-26266c4908d4\" (UID: \"91816a65-55c7-452d-ba71-26266c4908d4\") "
Jul 24 19:32:19.319712 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.319651    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "91816a65-55c7-452d-ba71-26266c4908d4" (UID: "91816a65-55c7-452d-ba71-26266c4908d4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:32:19.319902 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.319880    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91816a65-55c7-452d-ba71-26266c4908d4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "91816a65-55c7-452d-ba71-26266c4908d4" (UID: "91816a65-55c7-452d-ba71-26266c4908d4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:32:19.319971 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.319923    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "91816a65-55c7-452d-ba71-26266c4908d4" (UID: "91816a65-55c7-452d-ba71-26266c4908d4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:32:19.320132 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.319976    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "91816a65-55c7-452d-ba71-26266c4908d4" (UID: "91816a65-55c7-452d-ba71-26266c4908d4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:32:19.320387 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.320359    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-istio-data" (OuterVolumeSpecName: "istio-data") pod "91816a65-55c7-452d-ba71-26266c4908d4" (UID: "91816a65-55c7-452d-ba71-26266c4908d4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:32:19.321485 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.321457    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91816a65-55c7-452d-ba71-26266c4908d4-istio-token" (OuterVolumeSpecName: "istio-token") pod "91816a65-55c7-452d-ba71-26266c4908d4" (UID: "91816a65-55c7-452d-ba71-26266c4908d4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:32:19.321604 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.321586    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "91816a65-55c7-452d-ba71-26266c4908d4" (UID: "91816a65-55c7-452d-ba71-26266c4908d4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:32:19.322225 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.322196    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91816a65-55c7-452d-ba71-26266c4908d4-kube-api-access-njmp9" (OuterVolumeSpecName: "kube-api-access-njmp9") pod "91816a65-55c7-452d-ba71-26266c4908d4" (UID: "91816a65-55c7-452d-ba71-26266c4908d4"). InnerVolumeSpecName "kube-api-access-njmp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:32:19.322305 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.322265    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91816a65-55c7-452d-ba71-26266c4908d4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "91816a65-55c7-452d-ba71-26266c4908d4" (UID: "91816a65-55c7-452d-ba71-26266c4908d4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 19:32:19.419962 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.419939    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-workload-socket\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:19.419962 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.419963    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-istio-data\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:19.420178 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.419976    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91816a65-55c7-452d-ba71-26266c4908d4-istio-token\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:19.420178 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.419988    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91816a65-55c7-452d-ba71-26266c4908d4-istio-podinfo\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:19.420178 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.420001    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-credential-socket\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:19.420178 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.420015    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-workload-certs\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:19.420178 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.420027    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91816a65-55c7-452d-ba71-26266c4908d4-istiod-ca-cert\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:19.420178 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.420035    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njmp9\" (UniqueName: \"kubernetes.io/projected/91816a65-55c7-452d-ba71-26266c4908d4-kube-api-access-njmp9\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:19.420178 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.420044    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91816a65-55c7-452d-ba71-26266c4908d4-istio-envoy\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:19.421117 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.421089    2563 generic.go:356] "Generic (PLEG): container finished" podID="91816a65-55c7-452d-ba71-26266c4908d4" containerID="de2929d0cc8636b83d341a6b883ec2f68536f1dc17e03fb01729ec0ccea9390a" exitCode=0
Jul 24 19:32:19.421163 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.421143    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"
Jul 24 19:32:19.421195 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.421178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw" event={"ID":"91816a65-55c7-452d-ba71-26266c4908d4","Type":"ContainerDied","Data":"de2929d0cc8636b83d341a6b883ec2f68536f1dc17e03fb01729ec0ccea9390a"}
Jul 24 19:32:19.421242 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.421231    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw" event={"ID":"91816a65-55c7-452d-ba71-26266c4908d4","Type":"ContainerDied","Data":"04ddc363afa14cac1e2572211509712e77f6ffcc823dec9d2777f0e9eb42fd96"}
Jul 24 19:32:19.421280 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.421255    2563 scope.go:117] "RemoveContainer" containerID="de2929d0cc8636b83d341a6b883ec2f68536f1dc17e03fb01729ec0ccea9390a"
Jul 24 19:32:19.430768 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.430753    2563 scope.go:117] "RemoveContainer" containerID="9795d2cc7a7950150689e90e8d3a3e2bb895bce91a172f6084e7a76c04f5c903"
Jul 24 19:32:19.438945 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.438929    2563 scope.go:117] "RemoveContainer" containerID="de2929d0cc8636b83d341a6b883ec2f68536f1dc17e03fb01729ec0ccea9390a"
Jul 24 19:32:19.439185 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:32:19.439167    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de2929d0cc8636b83d341a6b883ec2f68536f1dc17e03fb01729ec0ccea9390a\": container with ID starting with de2929d0cc8636b83d341a6b883ec2f68536f1dc17e03fb01729ec0ccea9390a not found: ID does not exist" containerID="de2929d0cc8636b83d341a6b883ec2f68536f1dc17e03fb01729ec0ccea9390a"
Jul 24 19:32:19.439230 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.439194    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de2929d0cc8636b83d341a6b883ec2f68536f1dc17e03fb01729ec0ccea9390a"} err="failed to get container status \"de2929d0cc8636b83d341a6b883ec2f68536f1dc17e03fb01729ec0ccea9390a\": rpc error: code = NotFound desc = could not find container \"de2929d0cc8636b83d341a6b883ec2f68536f1dc17e03fb01729ec0ccea9390a\": container with ID starting with de2929d0cc8636b83d341a6b883ec2f68536f1dc17e03fb01729ec0ccea9390a not found: ID does not exist"
Jul 24 19:32:19.439230 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.439212    2563 scope.go:117] "RemoveContainer" containerID="9795d2cc7a7950150689e90e8d3a3e2bb895bce91a172f6084e7a76c04f5c903"
Jul 24 19:32:19.439466 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:32:19.439449    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9795d2cc7a7950150689e90e8d3a3e2bb895bce91a172f6084e7a76c04f5c903\": container with ID starting with 9795d2cc7a7950150689e90e8d3a3e2bb895bce91a172f6084e7a76c04f5c903 not found: ID does not exist" containerID="9795d2cc7a7950150689e90e8d3a3e2bb895bce91a172f6084e7a76c04f5c903"
Jul 24 19:32:19.439529 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.439471    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9795d2cc7a7950150689e90e8d3a3e2bb895bce91a172f6084e7a76c04f5c903"} err="failed to get container status \"9795d2cc7a7950150689e90e8d3a3e2bb895bce91a172f6084e7a76c04f5c903\": rpc error: code = NotFound desc = could not find container \"9795d2cc7a7950150689e90e8d3a3e2bb895bce91a172f6084e7a76c04f5c903\": container with ID starting with 9795d2cc7a7950150689e90e8d3a3e2bb895bce91a172f6084e7a76c04f5c903 not found: ID does not exist"
Jul 24 19:32:19.443031 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.443011    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"]
Jul 24 19:32:19.446525 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:19.446506    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-e05f4d-openshift-default-5b947854b4-pfthw"]
Jul 24 19:32:20.203843 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:20.203800    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91816a65-55c7-452d-ba71-26266c4908d4" path="/var/lib/kubelet/pods/91816a65-55c7-452d-ba71-26266c4908d4/volumes"
Jul 24 19:32:24.858189 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:24.858153    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"]
Jul 24 19:32:24.858698 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:24.858387    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72" podUID="364046bf-1994-4881-a3bf-44d5837b73d4" containerName="maas-api" containerID="cri-o://33684aeb12c5f00dc098b6f8c81e68c842139ab44cb3c6d233bdf6eb5caed8df" gracePeriod=30
Jul 24 19:32:24.925119 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:24.925086    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"]
Jul 24 19:32:24.925343 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:24.925309    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz" podUID="6ab47fd9-2876-43be-95bc-a6a3294c268a" containerName="payload-pre-processing" containerID="cri-o://e1c933dc4c57780b43c9ac307d21a4e05ab49fa092ccdf6a072e62b4efa851d6" gracePeriod=30
Jul 24 19:32:25.175075 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.175048    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:32:25.208802 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.208773    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"
Jul 24 19:32:25.262701 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.262668    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8xk2\" (UniqueName: \"kubernetes.io/projected/364046bf-1994-4881-a3bf-44d5837b73d4-kube-api-access-w8xk2\") pod \"364046bf-1994-4881-a3bf-44d5837b73d4\" (UID: \"364046bf-1994-4881-a3bf-44d5837b73d4\") "
Jul 24 19:32:25.262877 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.262713    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/364046bf-1994-4881-a3bf-44d5837b73d4-maas-api-tls\") pod \"364046bf-1994-4881-a3bf-44d5837b73d4\" (UID: \"364046bf-1994-4881-a3bf-44d5837b73d4\") "
Jul 24 19:32:25.262877 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.262731    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ab47fd9-2876-43be-95bc-a6a3294c268a-plugins-config-volume\") pod \"6ab47fd9-2876-43be-95bc-a6a3294c268a\" (UID: \"6ab47fd9-2876-43be-95bc-a6a3294c268a\") "
Jul 24 19:32:25.262877 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.262765    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9x4z9\" (UniqueName: \"kubernetes.io/projected/6ab47fd9-2876-43be-95bc-a6a3294c268a-kube-api-access-9x4z9\") pod \"6ab47fd9-2876-43be-95bc-a6a3294c268a\" (UID: \"6ab47fd9-2876-43be-95bc-a6a3294c268a\") "
Jul 24 19:32:25.262877 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.262816    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/364046bf-1994-4881-a3bf-44d5837b73d4-openshift-service-ca\") pod \"364046bf-1994-4881-a3bf-44d5837b73d4\" (UID: \"364046bf-1994-4881-a3bf-44d5837b73d4\") "
Jul 24 19:32:25.263149 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.263118    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ab47fd9-2876-43be-95bc-a6a3294c268a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6ab47fd9-2876-43be-95bc-a6a3294c268a" (UID: "6ab47fd9-2876-43be-95bc-a6a3294c268a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:32:25.263255 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.263229    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/364046bf-1994-4881-a3bf-44d5837b73d4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "364046bf-1994-4881-a3bf-44d5837b73d4" (UID: "364046bf-1994-4881-a3bf-44d5837b73d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:32:25.264990 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.264960    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/364046bf-1994-4881-a3bf-44d5837b73d4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "364046bf-1994-4881-a3bf-44d5837b73d4" (UID: "364046bf-1994-4881-a3bf-44d5837b73d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:32:25.265104 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.265036    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/364046bf-1994-4881-a3bf-44d5837b73d4-kube-api-access-w8xk2" (OuterVolumeSpecName: "kube-api-access-w8xk2") pod "364046bf-1994-4881-a3bf-44d5837b73d4" (UID: "364046bf-1994-4881-a3bf-44d5837b73d4"). InnerVolumeSpecName "kube-api-access-w8xk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:32:25.265104 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.265042    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ab47fd9-2876-43be-95bc-a6a3294c268a-kube-api-access-9x4z9" (OuterVolumeSpecName: "kube-api-access-9x4z9") pod "6ab47fd9-2876-43be-95bc-a6a3294c268a" (UID: "6ab47fd9-2876-43be-95bc-a6a3294c268a"). InnerVolumeSpecName "kube-api-access-9x4z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:32:25.363553 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.363518    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8xk2\" (UniqueName: \"kubernetes.io/projected/364046bf-1994-4881-a3bf-44d5837b73d4-kube-api-access-w8xk2\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:25.363553 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.363545    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/364046bf-1994-4881-a3bf-44d5837b73d4-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:25.363553 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.363557    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ab47fd9-2876-43be-95bc-a6a3294c268a-plugins-config-volume\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:25.363553 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.363566    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9x4z9\" (UniqueName: \"kubernetes.io/projected/6ab47fd9-2876-43be-95bc-a6a3294c268a-kube-api-access-9x4z9\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:25.363823 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.363574    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/364046bf-1994-4881-a3bf-44d5837b73d4-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\""
Jul 24 19:32:25.454976 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.454887    2563 generic.go:356] "Generic (PLEG): container finished" podID="364046bf-1994-4881-a3bf-44d5837b73d4" containerID="33684aeb12c5f00dc098b6f8c81e68c842139ab44cb3c6d233bdf6eb5caed8df" exitCode=0
Jul 24 19:32:25.454976 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.454929    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"
Jul 24 19:32:25.454976 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.454963    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72" event={"ID":"364046bf-1994-4881-a3bf-44d5837b73d4","Type":"ContainerDied","Data":"33684aeb12c5f00dc098b6f8c81e68c842139ab44cb3c6d233bdf6eb5caed8df"}
Jul 24 19:32:25.455243 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.455016    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72" event={"ID":"364046bf-1994-4881-a3bf-44d5837b73d4","Type":"ContainerDied","Data":"f5dc10a7b43b3b37b6e28dc5b2927fab0559afc6766ab4693c20079575f49079"}
Jul 24 19:32:25.455243 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.455047    2563 scope.go:117] "RemoveContainer" containerID="33684aeb12c5f00dc098b6f8c81e68c842139ab44cb3c6d233bdf6eb5caed8df"
Jul 24 19:32:25.456418 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.456375    2563 generic.go:356] "Generic (PLEG): container finished" podID="6ab47fd9-2876-43be-95bc-a6a3294c268a" containerID="e1c933dc4c57780b43c9ac307d21a4e05ab49fa092ccdf6a072e62b4efa851d6" exitCode=0
Jul 24 19:32:25.456545 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.456440    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"
Jul 24 19:32:25.456545 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.456434    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz" event={"ID":"6ab47fd9-2876-43be-95bc-a6a3294c268a","Type":"ContainerDied","Data":"e1c933dc4c57780b43c9ac307d21a4e05ab49fa092ccdf6a072e62b4efa851d6"}
Jul 24 19:32:25.456651 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.456558    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz" event={"ID":"6ab47fd9-2876-43be-95bc-a6a3294c268a","Type":"ContainerDied","Data":"1fa6697252a275d05f875e424fb942f3d0ee95c6bd4a74181e2a6dd83123f80b"}
Jul 24 19:32:25.464341 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.464323    2563 scope.go:117] "RemoveContainer" containerID="33684aeb12c5f00dc098b6f8c81e68c842139ab44cb3c6d233bdf6eb5caed8df"
Jul 24 19:32:25.464699 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:32:25.464671    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33684aeb12c5f00dc098b6f8c81e68c842139ab44cb3c6d233bdf6eb5caed8df\": container with ID starting with 33684aeb12c5f00dc098b6f8c81e68c842139ab44cb3c6d233bdf6eb5caed8df not found: ID does not exist" containerID="33684aeb12c5f00dc098b6f8c81e68c842139ab44cb3c6d233bdf6eb5caed8df"
Jul 24 19:32:25.464800 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.464723    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33684aeb12c5f00dc098b6f8c81e68c842139ab44cb3c6d233bdf6eb5caed8df"} err="failed to get container status \"33684aeb12c5f00dc098b6f8c81e68c842139ab44cb3c6d233bdf6eb5caed8df\": rpc error: code = NotFound desc = could not find container \"33684aeb12c5f00dc098b6f8c81e68c842139ab44cb3c6d233bdf6eb5caed8df\": container with ID starting with 33684aeb12c5f00dc098b6f8c81e68c842139ab44cb3c6d233bdf6eb5caed8df not found: ID does not exist"
Jul 24 19:32:25.464800 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.464748    2563 scope.go:117] "RemoveContainer" containerID="e1c933dc4c57780b43c9ac307d21a4e05ab49fa092ccdf6a072e62b4efa851d6"
Jul 24 19:32:25.472898 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.472878    2563 scope.go:117] "RemoveContainer" containerID="e1c933dc4c57780b43c9ac307d21a4e05ab49fa092ccdf6a072e62b4efa851d6"
Jul 24 19:32:25.473142 ip-10-0-141-213 kubenswrapper[2563]: E0724 19:32:25.473125    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1c933dc4c57780b43c9ac307d21a4e05ab49fa092ccdf6a072e62b4efa851d6\": container with ID starting with e1c933dc4c57780b43c9ac307d21a4e05ab49fa092ccdf6a072e62b4efa851d6 not found: ID does not exist" containerID="e1c933dc4c57780b43c9ac307d21a4e05ab49fa092ccdf6a072e62b4efa851d6"
Jul 24 19:32:25.473194 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.473150    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1c933dc4c57780b43c9ac307d21a4e05ab49fa092ccdf6a072e62b4efa851d6"} err="failed to get container status \"e1c933dc4c57780b43c9ac307d21a4e05ab49fa092ccdf6a072e62b4efa851d6\": rpc error: code = NotFound desc = could not find container \"e1c933dc4c57780b43c9ac307d21a4e05ab49fa092ccdf6a072e62b4efa851d6\": container with ID starting with e1c933dc4c57780b43c9ac307d21a4e05ab49fa092ccdf6a072e62b4efa851d6 not found: ID does not exist"
Jul 24 19:32:25.480352 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.480331    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"]
Jul 24 19:32:25.486536 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.486518    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-48f4d2-5967c94f8c-vfh72"]
Jul 24 19:32:25.496471 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.496443    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"]
Jul 24 19:32:25.500719 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:25.500700    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-48f4d2-856974fb4-6kmhz"]
Jul 24 19:32:26.203924 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:26.203842    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="364046bf-1994-4881-a3bf-44d5837b73d4" path="/var/lib/kubelet/pods/364046bf-1994-4881-a3bf-44d5837b73d4/volumes"
Jul 24 19:32:26.204279 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:26.204169    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ab47fd9-2876-43be-95bc-a6a3294c268a" path="/var/lib/kubelet/pods/6ab47fd9-2876-43be-95bc-a6a3294c268a/volumes"
Jul 24 19:32:27.654553 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:27.654521    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q4bw8"]
Jul 24 19:32:42.077624 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:42.077589    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-78b6l_8e360529-8929-437d-9d5b-c0c61d0c83a3/manager/0.log"
Jul 24 19:32:42.770997 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:42.770963    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6559d49d8b-nkx69_e976c7fd-067a-48d2-a06e-f27bce7c1aba/copy-manifests/0.log"
Jul 24 19:32:42.790002 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:42.789968    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6559d49d8b-nkx69_e976c7fd-067a-48d2-a06e-f27bce7c1aba/manager/0.log"
Jul 24 19:32:44.323188 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:44.323160    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs_9797f2d8-5332-4ea3-8033-422021d8bc0c/util/0.log"
Jul 24 19:32:44.328985 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:44.328960    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs_9797f2d8-5332-4ea3-8033-422021d8bc0c/pull/0.log"
Jul 24 19:32:44.334650 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:44.334630    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs_9797f2d8-5332-4ea3-8033-422021d8bc0c/extract/0.log"
Jul 24 19:32:44.445657 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:44.445634    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv_62188e01-a45c-4f10-bfba-24ecbf9c1000/util/0.log"
Jul 24 19:32:44.451838 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:44.451811    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv_62188e01-a45c-4f10-bfba-24ecbf9c1000/pull/0.log"
Jul 24 19:32:44.457786 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:44.457762    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv_62188e01-a45c-4f10-bfba-24ecbf9c1000/extract/0.log"
Jul 24 19:32:44.568472 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:44.568449    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f_511f5628-511a-4c8a-8d0b-88dc56223da3/util/0.log"
Jul 24 19:32:44.574828 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:44.574767    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f_511f5628-511a-4c8a-8d0b-88dc56223da3/pull/0.log"
Jul 24 19:32:44.580611 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:44.580590    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f_511f5628-511a-4c8a-8d0b-88dc56223da3/extract/0.log"
Jul 24 19:32:44.691117 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:44.691090    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv_10b4b2ad-c3c3-4909-86b0-61aec164facf/pull/0.log"
Jul 24 19:32:44.696929 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:44.696908    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv_10b4b2ad-c3c3-4909-86b0-61aec164facf/extract/0.log"
Jul 24 19:32:44.702093 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:44.702075    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv_10b4b2ad-c3c3-4909-86b0-61aec164facf/util/0.log"
Jul 24 19:32:45.492107 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:45.492023    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-q4bw8_b8246d36-7da8-4e55-a471-0232cc4eed49/limitador/0.log"
Jul 24 19:32:46.091732 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:46.091701    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-q6pr5_40d75a61-c418-46e0-8944-f5b8a38d48a2/discovery/0.log"
Jul 24 19:32:47.337386 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:47.337355    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-fjph6_2308a1d7-63df-4d83-8f98-0536d233d4ba/main/0.log"
Jul 24 19:32:55.547533 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:55.547491    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jwsd9_c844f7c8-ddea-4daa-9d99-b0f2b54ab7e3/global-pull-secret-syncer/0.log"
Jul 24 19:32:55.690261 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:55.690226    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9k9h7_3d0820a3-2459-4599-91e6-6e45d8ebb6f3/konnectivity-agent/0.log"
Jul 24 19:32:55.795125 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:55.795092    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-213.ec2.internal_a2ba8c9c359a1f1e65affdf9bc68bd46/haproxy/0.log"
Jul 24 19:32:59.697384 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:59.697352    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs_9797f2d8-5332-4ea3-8033-422021d8bc0c/extract/0.log"
Jul 24 19:32:59.720639 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:59.720613    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs_9797f2d8-5332-4ea3-8033-422021d8bc0c/util/0.log"
Jul 24 19:32:59.742048 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:59.742024    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpxrs_9797f2d8-5332-4ea3-8033-422021d8bc0c/pull/0.log"
Jul 24 19:32:59.780760 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:59.780722    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv_62188e01-a45c-4f10-bfba-24ecbf9c1000/extract/0.log"
Jul 24 19:32:59.810352 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:59.810330    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv_62188e01-a45c-4f10-bfba-24ecbf9c1000/util/0.log"
Jul 24 19:32:59.840066 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:59.840040    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07zjtv_62188e01-a45c-4f10-bfba-24ecbf9c1000/pull/0.log"
Jul 24 19:32:59.878547 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:59.878524    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f_511f5628-511a-4c8a-8d0b-88dc56223da3/extract/0.log"
Jul 24 19:32:59.904908 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:59.904883    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f_511f5628-511a-4c8a-8d0b-88dc56223da3/util/0.log"
Jul 24 19:32:59.930536 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:59.930513    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k9p5f_511f5628-511a-4c8a-8d0b-88dc56223da3/pull/0.log"
Jul 24 19:32:59.975155 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:59.975078    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv_10b4b2ad-c3c3-4909-86b0-61aec164facf/extract/0.log"
Jul 24 19:32:59.996308 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:32:59.996279    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv_10b4b2ad-c3c3-4909-86b0-61aec164facf/util/0.log"
Jul 24 19:33:00.028446 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:00.028406    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14qrzv_10b4b2ad-c3c3-4909-86b0-61aec164facf/pull/0.log"
Jul 24 19:33:00.769285 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:00.769257    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-q4bw8_b8246d36-7da8-4e55-a471-0232cc4eed49/limitador/0.log"
Jul 24 19:33:02.615133 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:02.615100    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_37b0381f-1920-4032-95d3-e3f38ffc4149/alertmanager/0.log"
Jul 24 19:33:02.641560 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:02.641533    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_37b0381f-1920-4032-95d3-e3f38ffc4149/config-reloader/0.log"
Jul 24 19:33:02.664797 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:02.664772    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_37b0381f-1920-4032-95d3-e3f38ffc4149/kube-rbac-proxy-web/0.log"
Jul 24 19:33:02.699026 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:02.699001    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_37b0381f-1920-4032-95d3-e3f38ffc4149/kube-rbac-proxy/0.log"
Jul 24 19:33:02.740352 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:02.740324    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_37b0381f-1920-4032-95d3-e3f38ffc4149/kube-rbac-proxy-metric/0.log"
Jul 24 19:33:02.760991 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:02.760961    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_37b0381f-1920-4032-95d3-e3f38ffc4149/prom-label-proxy/0.log"
Jul 24 19:33:02.787733 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:02.787709    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_37b0381f-1920-4032-95d3-e3f38ffc4149/init-config-reloader/0.log"
Jul 24 19:33:03.212698 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:03.212668    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cxgj5_076e1bdd-08e1-4c23-8029-91474ae2132a/node-exporter/0.log"
Jul 24 19:33:03.241628 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:03.241599    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cxgj5_076e1bdd-08e1-4c23-8029-91474ae2132a/kube-rbac-proxy/0.log"
Jul 24 19:33:03.273236 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:03.273203    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cxgj5_076e1bdd-08e1-4c23-8029-91474ae2132a/init-textfile/0.log"
Jul 24 19:33:06.386663 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.386621    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"]
Jul 24 19:33:06.387118 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387030    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4503a25-3dac-40ac-9717-627c86ad4d6f" containerName="payload-processing"
Jul 24 19:33:06.387118 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387048    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4503a25-3dac-40ac-9717-627c86ad4d6f" containerName="payload-processing"
Jul 24 19:33:06.387118 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387075    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78a8e8a6-da1d-4685-b288-0092732f0458" containerName="payload-processing"
Jul 24 19:33:06.387118 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387083    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="78a8e8a6-da1d-4685-b288-0092732f0458" containerName="payload-processing"
Jul 24 19:33:06.387118 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387093    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0d0bdef-5d75-4a72-8132-c2b076f17a85" containerName="payload-pre-processing"
Jul 24 19:33:06.387118 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387101    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0d0bdef-5d75-4a72-8132-c2b076f17a85" containerName="payload-pre-processing"
Jul 24 19:33:06.387118 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387118    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4124d1fa-edba-4079-a170-b30c203f545e" containerName="maas-api"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387126    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4124d1fa-edba-4079-a170-b30c203f545e" containerName="maas-api"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387141    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91816a65-55c7-452d-ba71-26266c4908d4" containerName="istio-proxy"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387149    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="91816a65-55c7-452d-ba71-26266c4908d4" containerName="istio-proxy"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387158    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa0f22d5-9afe-4d04-bbe8-b63075f9bd85" containerName="maas-api"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387173    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa0f22d5-9afe-4d04-bbe8-b63075f9bd85" containerName="maas-api"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387190    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="364046bf-1994-4881-a3bf-44d5837b73d4" containerName="maas-api"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387198    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="364046bf-1994-4881-a3bf-44d5837b73d4" containerName="maas-api"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387212    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ab47fd9-2876-43be-95bc-a6a3294c268a" containerName="payload-pre-processing"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387220    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ab47fd9-2876-43be-95bc-a6a3294c268a" containerName="payload-pre-processing"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387234    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="842bf3a4-5e60-4045-a932-6f569740ee01" containerName="maas-api"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387242    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="842bf3a4-5e60-4045-a932-6f569740ee01" containerName="maas-api"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387251    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bd845bf-485c-4ce3-8d89-79b7a99dcf71" containerName="payload-processing"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387259    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bd845bf-485c-4ce3-8d89-79b7a99dcf71" containerName="payload-processing"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387340    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4503a25-3dac-40ac-9717-627c86ad4d6f" containerName="payload-processing"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387352    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="91816a65-55c7-452d-ba71-26266c4908d4" containerName="istio-proxy"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387362    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="364046bf-1994-4881-a3bf-44d5837b73d4" containerName="maas-api"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387370    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4124d1fa-edba-4079-a170-b30c203f545e" containerName="maas-api"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387379    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0d0bdef-5d75-4a72-8132-c2b076f17a85" containerName="payload-pre-processing"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387406    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa0f22d5-9afe-4d04-bbe8-b63075f9bd85" containerName="maas-api"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387415    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="842bf3a4-5e60-4045-a932-6f569740ee01" containerName="maas-api"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387426    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="78a8e8a6-da1d-4685-b288-0092732f0458" containerName="payload-processing"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387434    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ab47fd9-2876-43be-95bc-a6a3294c268a" containerName="payload-pre-processing"
Jul 24 19:33:06.387620 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.387441    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bd845bf-485c-4ce3-8d89-79b7a99dcf71" containerName="payload-processing"
Jul 24 19:33:06.390158 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.390140    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.392463 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.392439    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rq44j\"/\"openshift-service-ca.crt\""
Jul 24 19:33:06.392575 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.392522    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rq44j\"/\"default-dockercfg-c7tk9\""
Jul 24 19:33:06.393550 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.393534    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rq44j\"/\"kube-root-ca.crt\""
Jul 24 19:33:06.396206 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.396185    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"]
Jul 24 19:33:06.504801 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.504765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4f3665c-ac85-4540-bfe9-7e6d937cf998-lib-modules\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.504801 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.504802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f4f3665c-ac85-4540-bfe9-7e6d937cf998-podres\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.505030 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.504836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f4f3665c-ac85-4540-bfe9-7e6d937cf998-proc\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.505030 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.504897    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4f3665c-ac85-4540-bfe9-7e6d937cf998-sys\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.505030 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.504931    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxqht\" (UniqueName: \"kubernetes.io/projected/f4f3665c-ac85-4540-bfe9-7e6d937cf998-kube-api-access-kxqht\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.606218 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.606182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4f3665c-ac85-4540-bfe9-7e6d937cf998-sys\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.606447 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.606227    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxqht\" (UniqueName: \"kubernetes.io/projected/f4f3665c-ac85-4540-bfe9-7e6d937cf998-kube-api-access-kxqht\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.606447 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.606320    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4f3665c-ac85-4540-bfe9-7e6d937cf998-sys\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.606447 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.606319    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4f3665c-ac85-4540-bfe9-7e6d937cf998-lib-modules\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.606447 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.606379    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f4f3665c-ac85-4540-bfe9-7e6d937cf998-podres\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.606646 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.606452    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4f3665c-ac85-4540-bfe9-7e6d937cf998-lib-modules\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.606646 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.606462    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f4f3665c-ac85-4540-bfe9-7e6d937cf998-proc\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.606646 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.606507    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f4f3665c-ac85-4540-bfe9-7e6d937cf998-proc\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.606646 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.606547    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f4f3665c-ac85-4540-bfe9-7e6d937cf998-podres\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.613538 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.613518    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxqht\" (UniqueName: \"kubernetes.io/projected/f4f3665c-ac85-4540-bfe9-7e6d937cf998-kube-api-access-kxqht\") pod \"perf-node-gather-daemonset-dzkh5\" (UID: \"f4f3665c-ac85-4540-bfe9-7e6d937cf998\") " pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.700367 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.700275    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:06.826890 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:06.826851    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"]
Jul 24 19:33:06.828787 ip-10-0-141-213 kubenswrapper[2563]: W0724 19:33:06.828748    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf4f3665c_ac85_4540_bfe9_7e6d937cf998.slice/crio-048b93bafb817688922bc71bd9bb03a8546c3ac22b12b0ce39cb912c664455c9 WatchSource:0}: Error finding container 048b93bafb817688922bc71bd9bb03a8546c3ac22b12b0ce39cb912c664455c9: Status 404 returned error can't find the container with id 048b93bafb817688922bc71bd9bb03a8546c3ac22b12b0ce39cb912c664455c9
Jul 24 19:33:07.661789 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:07.661751    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5" event={"ID":"f4f3665c-ac85-4540-bfe9-7e6d937cf998","Type":"ContainerStarted","Data":"7c88bb82e07ae3959064d75c9624233d3bd26e22a3264672ef5c82fdeed139b1"}
Jul 24 19:33:07.662176 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:07.661829    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5" event={"ID":"f4f3665c-ac85-4540-bfe9-7e6d937cf998","Type":"ContainerStarted","Data":"048b93bafb817688922bc71bd9bb03a8546c3ac22b12b0ce39cb912c664455c9"}
Jul 24 19:33:07.662176 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:07.661891    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:07.679775 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:07.679725    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5" podStartSLOduration=1.679712608 podStartE2EDuration="1.679712608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 19:33:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:33:07.67749192 +0000 UTC m=+2926.068443394" watchObservedRunningTime="2026-07-24 19:33:07.679712608 +0000 UTC m=+2926.070664084"
Jul 24 19:33:08.590505 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:08.590475    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nrlxk_e65a79e4-e43c-43a5-8a73-bff5b0a4d8de/dns/0.log"
Jul 24 19:33:08.612995 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:08.612963    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nrlxk_e65a79e4-e43c-43a5-8a73-bff5b0a4d8de/kube-rbac-proxy/0.log"
Jul 24 19:33:08.742064 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:08.742036    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qmjbp_fe6a8da4-b859-441c-9001-7b14d9c8f065/dns-node-resolver/0.log"
Jul 24 19:33:09.326090 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:09.326061    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n89pr_a8f329d8-6209-4a6c-8004-60a1514c92ec/node-ca/0.log"
Jul 24 19:33:10.591117 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:10.591083    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-q6pr5_40d75a61-c418-46e0-8944-f5b8a38d48a2/discovery/0.log"
Jul 24 19:33:11.668976 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:11.668947    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fljrs_a3e1f504-4766-4c99-b3b1-670a846755d7/serve-healthcheck-canary/0.log"
Jul 24 19:33:12.309147 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:12.309115    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mnpws_1a81134d-1a8b-44e8-8d59-0346a25f0366/kube-rbac-proxy/0.log"
Jul 24 19:33:12.330607 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:12.330567    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mnpws_1a81134d-1a8b-44e8-8d59-0346a25f0366/exporter/0.log"
Jul 24 19:33:12.354762 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:12.354736    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mnpws_1a81134d-1a8b-44e8-8d59-0346a25f0366/extractor/0.log"
Jul 24 19:33:13.675845 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:13.675815    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rq44j/perf-node-gather-daemonset-dzkh5"
Jul 24 19:33:14.854528 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:14.854494    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-78b6l_8e360529-8929-437d-9d5b-c0c61d0c83a3/manager/0.log"
Jul 24 19:33:15.336174 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:15.336139    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6559d49d8b-nkx69_e976c7fd-067a-48d2-a06e-f27bce7c1aba/manager/0.log"
Jul 24 19:33:15.353782 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:15.353754    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6559d49d8b-nkx69_e976c7fd-067a-48d2-a06e-f27bce7c1aba/copy-manifests/0.log"
Jul 24 19:33:16.743061 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:16.743030    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-594d74945-klg86_6cc2476e-7fea-4b2b-ab47-9040834dba55/manager/0.log"
Jul 24 19:33:16.767716 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:16.767687    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-dgwwl_0c63da4d-799c-41e9-bfb0-44a78d4dfbc8/openshift-lws-operator/0.log"
Jul 24 19:33:17.428838 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:17.428805    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-q6pr5_40d75a61-c418-46e0-8944-f5b8a38d48a2/discovery/0.log"
Jul 24 19:33:19.245590 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:19.245551    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-q6pr5_40d75a61-c418-46e0-8944-f5b8a38d48a2/discovery/0.log"
Jul 24 19:33:23.755387 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:23.755357    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tpk4s_5eda199b-9a46-4bdd-919e-dbacd4631428/kube-multus-additional-cni-plugins/0.log"
Jul 24 19:33:23.775213 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:23.775180    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tpk4s_5eda199b-9a46-4bdd-919e-dbacd4631428/egress-router-binary-copy/0.log"
Jul 24 19:33:23.799903 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:23.799871    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tpk4s_5eda199b-9a46-4bdd-919e-dbacd4631428/cni-plugins/0.log"
Jul 24 19:33:23.815832 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:23.815805    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tpk4s_5eda199b-9a46-4bdd-919e-dbacd4631428/bond-cni-plugin/0.log"
Jul 24 19:33:23.834667 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:23.834636    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tpk4s_5eda199b-9a46-4bdd-919e-dbacd4631428/routeoverride-cni/0.log"
Jul 24 19:33:23.853259 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:23.853235    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tpk4s_5eda199b-9a46-4bdd-919e-dbacd4631428/whereabouts-cni-bincopy/0.log"
Jul 24 19:33:23.875425 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:23.875377    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tpk4s_5eda199b-9a46-4bdd-919e-dbacd4631428/whereabouts-cni/0.log"
Jul 24 19:33:23.964043 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:23.964008    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fmsn5_26bb3600-39ee-4ac3-9baa-b6fc520712a2/kube-multus/0.log"
Jul 24 19:33:24.124579 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:24.124548    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zgg7r_7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19/network-metrics-daemon/0.log"
Jul 24 19:33:24.142521 ip-10-0-141-213 kubenswrapper[2563]: I0724 19:33:24.142496    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zgg7r_7c5cdbc6-aede-4ffe-8fec-e5bd6c2ced19/kube-rbac-proxy/0.log"