Sep 11 16:27:45.299934 ip-10-0-135-58 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 16:27:45.299945 ip-10-0-135-58 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 16:27:45.299954 ip-10-0-135-58 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 16:27:45.300260 ip-10-0-135-58 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 16:27:55.342693 ip-10-0-135-58 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 16:27:55.342706 ip-10-0-135-58 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 3c57ce95d2c5407eac7209cb51c35fd1 --
Sep 11 16:29:40.249109 ip-10-0-135-58 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 16:29:40.680317 ip-10-0-135-58 kubenswrapper[2545]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 16:29:40.680317 ip-10-0-135-58 kubenswrapper[2545]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 16:29:40.680317 ip-10-0-135-58 kubenswrapper[2545]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 16:29:40.680317 ip-10-0-135-58 kubenswrapper[2545]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 16:29:40.680317 ip-10-0-135-58 kubenswrapper[2545]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 16:29:40.682907 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.682812    2545 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 16:29:40.687181 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687166    2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:29:40.687181 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687182    2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687185    2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687189    2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687193    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687197    2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687200    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687202    2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687205    2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687208    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687211    2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687214    2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687217    2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687219    2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687222    2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687225    2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687228    2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687230    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687233    2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687236    2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687239    2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:29:40.687245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687241    2545 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687244    2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687246    2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687249    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687252    2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687254    2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687264    2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687266    2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687269    2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687271    2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687274    2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687277    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687279    2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687282    2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687284    2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687287    2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687289    2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687291    2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687294    2545 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687297    2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:29:40.687715 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687299    2545 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687301    2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687304    2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687306    2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687309    2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687311    2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687320    2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687323    2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687325    2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687328    2545 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687330    2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687333    2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687336    2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687338    2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687341    2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687344    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687347    2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687349    2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687352    2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687360    2545 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:29:40.688208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687363    2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687367    2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687371    2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687374    2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687376    2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687379    2545 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687382    2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687385    2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687387    2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687390    2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687393    2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687396    2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687398    2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687401    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687403    2545 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687406    2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687414    2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687417    2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687421    2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:29:40.688701 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687424    2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687426    2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687429    2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687432    2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687436    2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687439    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687442    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687446    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687449    2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687452    2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687454    2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687457    2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687459    2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687468    2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687471    2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687474    2545 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687476    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687479    2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.687481    2545 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:29:40.689267 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689273    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689281    2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689285    2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689289    2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689293    2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689296    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689299    2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689303    2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689307    2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689310    2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689313    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689315    2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689318    2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689321    2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689324    2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689326    2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689329    2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689331    2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689334    2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:29:40.689733 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689336    2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689339    2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689341    2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689344    2545 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689346    2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689348    2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689351    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689353    2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689357    2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689359    2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689362    2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689364    2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689366    2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689369    2545 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689371    2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689375    2545 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689378    2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689381    2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689383    2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689385    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:29:40.690224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689388    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689392    2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689395    2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689398    2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689401    2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689403    2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689406    2545 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689409    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689411    2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689415    2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689417    2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689420    2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689423    2545 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689425    2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689428    2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689431    2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689433    2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689435    2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689438    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:29:40.690713 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689440    2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689443    2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689446    2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689448    2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689451    2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689454    2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689456    2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689459    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689462    2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689465    2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689467    2545 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689469    2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689472    2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689474    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689477    2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689479    2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689482    2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689485    2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689488    2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689490    2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:29:40.691204 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689493    2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689496    2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689498    2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689501    2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689503    2545 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689506    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689508    2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689511    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689513    2545 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689516    2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689518    2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689520    2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689523    2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689526    2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689528    2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689531    2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689533    2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689535    2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689538    2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689540    2545 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:29:40.691695 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689543    2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689923    2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689929    2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689931    2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689934    2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689937    2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689939    2545 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689942    2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689945    2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689949    2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689952    2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689955    2545 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689959    2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689962    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689964    2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689967    2545 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689970    2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689972    2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689976    2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689978    2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:29:40.692198 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689981    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689984    2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689986    2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689989    2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689992    2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689994    2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.689997    2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690000    2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690002    2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690004    2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690007    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690009    2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690015    2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690018    2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690021    2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690024    2545 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690026    2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690029    2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690031    2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:29:40.692694 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690034    2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690038    2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690041    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690044    2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690047    2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690049    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690052    2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690055    2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690057    2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690060    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690063    2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690065    2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690068    2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690071    2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690073    2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690076    2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690080    2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690083    2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690086    2545 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:29:40.693174 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690088    2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690091    2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690094    2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690097    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690099    2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690101    2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690105    2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690108    2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690110    2545 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690113    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690115    2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690118    2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690120    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690123    2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690125    2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690127    2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690130    2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690132    2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690135    2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690137    2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:29:40.693644 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690141    2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690144    2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690146    2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690149    2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690151    2545 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690154    2545 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690156    2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690159    2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690162    2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690164    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690168    2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690170    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690172    2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690175    2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690178    2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690181    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690183    2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690186    2545 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690190    2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690193    2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:29:40.694175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690195    2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690198    2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690268    2545 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690310    2545 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690317    2545 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690321    2545 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690325    2545 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690328    2545 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690332    2545 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690336    2545 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690339    2545 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690342    2545 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690345    2545 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690348    2545 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690351    2545 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690354    2545 flags.go:64] FLAG: --cgroup-root=""
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690357    2545 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690359    2545 flags.go:64] FLAG: --client-ca-file=""
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690363    2545 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690366    2545 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690370    2545 flags.go:64] FLAG: --cluster-domain=""
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690372    2545 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690375    2545 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690378    2545 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 16:29:40.694671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690382    2545 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690385    2545 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690389    2545 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690391    2545 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690395    2545 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690398    2545 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690402    2545 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690404    2545 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690407    2545 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690410    2545 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690414    2545 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690417    2545 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690419    2545 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690422    2545 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690424    2545 flags.go:64] FLAG: --enable-server="true"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690427    2545 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690431    2545 flags.go:64] FLAG: --event-burst="100"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690434    2545 flags.go:64] FLAG: --event-qps="50"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690437    2545 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690442    2545 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690445    2545 flags.go:64] FLAG: --eviction-hard=""
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690448    2545 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690451    2545 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690454    2545 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690457    2545 flags.go:64] FLAG: --eviction-soft=""
Sep 11 16:29:40.695266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690459    2545 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690462    2545 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690465    2545 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690467    2545 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690470    2545 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690472    2545 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690475    2545 flags.go:64] FLAG: --feature-gates=""
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690479    2545 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690482    2545 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690484    2545 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690487    2545 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690490    2545 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690493    2545 flags.go:64] FLAG: --help="false"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690497    2545 flags.go:64] FLAG: --hostname-override="ip-10-0-135-58.ec2.internal"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690500    2545 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690503    2545 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690506    2545 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690509    2545 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690512    2545 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690515    2545 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690517    2545 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690520    2545 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690523    2545 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 16:29:40.695868 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690525    2545 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690528    2545 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690531    2545 flags.go:64] FLAG: --kube-reserved=""
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690535    2545 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690537    2545 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690540    2545 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690543    2545 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690545    2545 flags.go:64] FLAG: --lock-file=""
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690548    2545 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690551    2545 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690554    2545 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690558    2545 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690561    2545 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690565    2545 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690567    2545 flags.go:64] FLAG: --logging-format="text"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690570    2545 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690573    2545 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690576    2545 flags.go:64] FLAG: --manifest-url=""
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690579    2545 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690583    2545 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690586    2545 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690590    2545 flags.go:64] FLAG: --max-pods="110"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690593    2545 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690596    2545 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690598    2545 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 16:29:40.696449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690602    2545 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690604    2545 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690607    2545 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690610    2545 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690616    2545 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690619    2545 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690622    2545 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690625    2545 flags.go:64] FLAG: --pod-cidr=""
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690628    2545 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690632    2545 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690638    2545 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690641    2545 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690644    2545 flags.go:64] FLAG: --port="10250"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690647    2545 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690649    2545 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0af7faebe10689125"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690652    2545 flags.go:64] FLAG: --qos-reserved=""
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690655    2545 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690658    2545 flags.go:64] FLAG: --register-node="true"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690661    2545 flags.go:64] FLAG: --register-with-taints=""
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690664    2545 flags.go:64] FLAG: --registry-burst="10"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690667    2545 flags.go:64] FLAG: --registry-qps="5"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690670    2545 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690672    2545 flags.go:64] FLAG: --reserved-memory=""
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690676    2545 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690678    2545 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 16:29:40.697075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690681    2545 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690684    2545 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690686    2545 flags.go:64] FLAG: --runonce="false"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690689    2545 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690692    2545 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690696    2545 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690698    2545 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690701    2545 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690704    2545 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690706    2545 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690709    2545 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690712    2545 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690715    2545 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690717    2545 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690720    2545 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690723    2545 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690726    2545 flags.go:64] FLAG: --system-cgroups=""
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690731    2545 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690735    2545 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690738    2545 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690741    2545 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690745    2545 flags.go:64] FLAG: --tls-min-version=""
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690748    2545 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690750    2545 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690753    2545 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 16:29:40.697707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690755    2545 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690758    2545 flags.go:64] FLAG: --v="2"
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690762    2545 flags.go:64] FLAG: --version="false"
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690766    2545 flags.go:64] FLAG: --vmodule=""
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690769    2545 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.690772    2545 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690899    2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690904    2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690907    2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690909    2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690912    2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690914    2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690919    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690924    2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690927    2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690930    2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690932    2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690935    2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690938    2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690941    2545 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690943    2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:29:40.698346 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690946    2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690948    2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690951    2545 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690955    2545 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690958    2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690962    2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690966    2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690970    2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690973    2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690976    2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690979    2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690982    2545 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690984    2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690987    2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690989    2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690992    2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690994    2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.690997    2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691000    2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:29:40.698972 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691002    2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691005    2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691007    2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691010    2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691013    2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691016    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691019    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691021    2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691024    2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691026    2545 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691029    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691032    2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691034    2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691037    2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691039    2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691042    2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691046    2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691048    2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691052    2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:29:40.699714 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691055    2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691058    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691061    2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691063    2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691066    2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691068    2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691071    2545 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691073    2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691075    2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691078    2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691080    2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691083    2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691085    2545 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691088    2545 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691090    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691093    2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691095    2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691099    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691101    2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691104    2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:29:40.700298 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691107    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691109    2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691112    2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691114    2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691117    2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691120    2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691123    2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691125    2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691127    2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691131    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691133    2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691136    2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691138    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691140    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691143    2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691145    2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691148    2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691151    2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691153    2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691156    2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:29:40.700806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691158    2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:29:40.701339 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691160    2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:29:40.701339 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691163    2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:29:40.701339 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691166    2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:29:40.701339 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691168    2545 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:29:40.701339 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.691171    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:29:40.701339 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.692043    2545 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 16:29:40.701339 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.701238    2545 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 16:29:40.701339 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.701254    2545 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 16:29:40.702162 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.702151    2545 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 16:29:40.702192 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.702165    2545 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 16:29:40.702250 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702242    2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:29:40.702250 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702250    2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702254    2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702257    2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702260    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702264    2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702267    2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702269    2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702272    2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702274    2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702277    2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702280    2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702283    2545 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702285    2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702288    2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702291    2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702293    2545 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702296    2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702299    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702301    2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702305    2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:29:40.702308 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702308    2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702311    2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702314    2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702317    2545 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702320    2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702323    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702325    2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702328    2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702331    2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702334    2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702336    2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702339    2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702342    2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702344    2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702347    2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702349    2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702352    2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702354    2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702357    2545 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702359    2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:29:40.702792 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702361    2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702365    2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702369    2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702372    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702375    2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702379    2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702383    2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702385    2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702388    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702391    2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702393    2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702396    2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702398    2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702401    2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702404    2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702407    2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702410    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702413    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702415    2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:29:40.703291 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702418    2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702421    2545 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702423    2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702426    2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702428    2545 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702430    2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702433    2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702436    2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702438    2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702441    2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702443    2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702446    2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702449    2545 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702452    2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702454    2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702457    2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702459    2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702462    2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702464    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:29:40.703745 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702469    2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702472    2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702475    2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702478    2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702481    2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702483    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702485    2545 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702488    2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702491    2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702494    2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702496    2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702499    2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702501    2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702504    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702506    2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702509    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702512    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702514    2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702517    2545 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:29:40.704287 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702519    2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.702524    2545 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702628    2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702633    2545 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702636    2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702640    2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702645    2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702648    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702652    2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702655    2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702658    2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702661    2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702664    2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702666    2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702669    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:29:40.704763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702672    2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702675    2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702678    2545 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702680    2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702683    2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702685    2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702688    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702691    2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702694    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702696    2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702699    2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702701    2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702704    2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702706    2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702709    2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702711    2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702714    2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702716    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702719    2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702721    2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:29:40.705150 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702724    2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702726    2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702729    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702731    2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702733    2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702736    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702740    2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702743    2545 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702746    2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702749    2545 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702752    2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702755    2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702759    2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702762    2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702765    2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702767    2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702770    2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702773    2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702776    2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702779    2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:29:40.705630 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702781    2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702784    2545 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702786    2545 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702789    2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702791    2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702793    2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702796    2545 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702799    2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702801    2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702804    2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702807    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702809    2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702812    2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702827    2545 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702830    2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702833    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702835    2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702839    2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702842    2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702845    2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:29:40.706129 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702848    2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702851    2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702854    2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702857    2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702860    2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702869    2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702872    2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702874    2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702877    2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702879    2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702882    2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702885    2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702887    2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702890    2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702893    2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702895    2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702898    2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702901    2545 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702903    2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702906    2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:29:40.706614 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702908    2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:29:40.707097 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702911    2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:29:40.707097 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702913    2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:29:40.707097 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702916    2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:29:40.707097 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702918    2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:29:40.707097 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:40.702921    2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:29:40.707097 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.702926    2545 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 16:29:40.707097 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.703032    2545 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 16:29:40.707097 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.706487    2545 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 16:29:40.707511 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.707499    2545 server.go:1019] "Starting client certificate rotation"
Sep 11 16:29:40.707617 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.707601    2545 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 16:29:40.707648 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.707635    2545 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 16:29:40.710306 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.710293    2545 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 16:29:40.714699 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.714684    2545 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 16:29:40.725744 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.725725    2545 log.go:25] "Validated CRI v1 runtime API"
Sep 11 16:29:40.728993 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.728980    2545 log.go:25] "Validated CRI v1 image API"
Sep 11 16:29:40.729804 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.729790    2545 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 16:29:40.734331 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.734315    2545 fs.go:135] Filesystem UUIDs: map[0808ed14-9497-4b33-a81c-d7f61268aa89:/dev/nvme0n1p3 0cd94f9a-a403-4da0-a625-25d5a4632d71:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 11 16:29:40.734388 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.734332    2545 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 16:29:40.737276 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.737260    2545 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 16:29:40.740501 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.740217    2545 manager.go:216] Machine: {Timestamp:2026-09-11 16:29:40.738379943 +0000 UTC m=+0.372594475 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:2499996 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24610062dd8d6f2fd19ff263bd5b2f SystemUUID:ec246100-62dd-8d6f-2fd1-9ff263bd5b2f BootID:3c57ce95-d2c5-407e-ac72-09cb51c35fd1 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1b:c2:bd:89:e5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1b:c2:bd:89:e5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:7f:18:e0:dd:ae Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 16:29:40.740501 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.740489    2545 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 16:29:40.740646 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.740566    2545 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 16:29:40.744160 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.744127    2545 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 16:29:40.744317 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.744163    2545 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-58.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 16:29:40.745765 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.745753    2545 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 16:29:40.745839 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.745771    2545 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 16:29:40.745839 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.745782    2545 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 16:29:40.746667 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.746655    2545 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 16:29:40.747871 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.747860    2545 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 16:29:40.752444 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.752430    2545 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 16:29:40.752571 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.752561    2545 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 16:29:40.752637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.752597    2545 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 16:29:40.752637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.752611    2545 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 16:29:40.752637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.752625    2545 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 16:29:40.752637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.752638    2545 kubelet.go:398] "Adding apiserver pod source"
Sep 11 16:29:40.752802 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.752651    2545 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 16:29:40.753459 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.753031    2545 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 16:29:40.753459 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.753051    2545 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 16:29:40.755516 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.755497    2545 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 16:29:40.758747 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.758734    2545 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 16:29:40.758802 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.758764    2545 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 16:29:40.760681 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.760657    2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 16:29:40.760760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.760691    2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 16:29:40.760760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.760699    2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 16:29:40.760760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.760704    2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 16:29:40.760760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.760712    2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 16:29:40.760760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.760717    2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 16:29:40.760760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.760730    2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 16:29:40.760760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.760735    2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 16:29:40.760760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.760741    2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 16:29:40.760760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.760747    2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 16:29:40.761178 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.760786    2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 16:29:40.761178 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.760792    2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 16:29:40.761699 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.761687    2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 16:29:40.761699 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.761696    2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 16:29:40.763631 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.763602    2545 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 16:29:40.763687 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.763605    2545 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-58.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 16:29:40.765399 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.765387    2545 server.go:1268] "Started kubelet"
Sep 11 16:29:40.765471 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.765449    2545 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 16:29:40.765871 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.765809    2545 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 16:29:40.765951 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.765901    2545 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 16:29:40.766182 ip-10-0-135-58 systemd[1]: Started Kubernetes Kubelet.
Sep 11 16:29:40.767541 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.767526    2545 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 16:29:40.767792 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.767776    2545 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 16:29:40.771401 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.771379    2545 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 16:29:40.773097 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.773082    2545 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-58.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 16:29:40.774095 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.772972    2545 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-58.ec2.internal.18d451273279f843  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-58.ec2.internal,UID:ip-10-0-135-58.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-58.ec2.internal,},FirstTimestamp:2026-09-11 16:29:40.765358147 +0000 UTC m=+0.399572664,LastTimestamp:2026-09-11 16:29:40.765358147 +0000 UTC m=+0.399572664,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-58.ec2.internal,}"
Sep 11 16:29:40.774388 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.774373    2545 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 16:29:40.775081 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.775063    2545 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 16:29:40.775297 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.775277    2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-58.ec2.internal\" not found"
Sep 11 16:29:40.775803 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.775786    2545 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 16:29:40.775803 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.775791    2545 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 16:29:40.775951 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.775811    2545 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 16:29:40.775951 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.775942    2545 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 16:29:40.775951 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.775951    2545 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 16:29:40.776119 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.776104    2545 factory.go:55] Registering systemd factory
Sep 11 16:29:40.776176 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.776151    2545 factory.go:223] Registration of the systemd container factory successfully
Sep 11 16:29:40.776357 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.776344    2545 factory.go:153] Registering CRI-O factory
Sep 11 16:29:40.776357 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.776358    2545 factory.go:223] Registration of the crio container factory successfully
Sep 11 16:29:40.776466 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.776416    2545 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 16:29:40.776466 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.776429    2545 factory.go:103] Registering Raw factory
Sep 11 16:29:40.776466 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.776439    2545 manager.go:1196] Started watching for new ooms in manager
Sep 11 16:29:40.776869 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.776854    2545 manager.go:349] Starting recovery of all containers
Sep 11 16:29:40.779304 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.779275    2545 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-58.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 16:29:40.779461 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.779433    2545 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 16:29:40.789947 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.789931    2545 manager.go:354] Recovery completed
Sep 11 16:29:40.794024 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.794012    2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:29:40.795661 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.795642    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:29:40.795718 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.795677    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:29:40.795718 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.795694    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:29:40.796234 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.796222    2545 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 16:29:40.796234 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.796231    2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:29:40.796345 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.796233    2545 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 16:29:40.796345 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.796301    2545 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 16:29:40.796864 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.796852    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:29:40.796913 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.796876    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:29:40.796913 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.796888    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:29:40.797929 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.797914    2545 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g2t6v"
Sep 11 16:29:40.798537 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.798507    2545 policy_none.go:49] "None policy: Start"
Sep 11 16:29:40.798596 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.798542    2545 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 16:29:40.798596 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.798573    2545 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 16:29:40.798795 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.798734    2545 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-58.ec2.internal.18d45127344864a5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-58.ec2.internal,UID:ip-10-0-135-58.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-58.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-58.ec2.internal,},FirstTimestamp:2026-09-11 16:29:40.795663525 +0000 UTC m=+0.429878048,LastTimestamp:2026-09-11 16:29:40.795663525 +0000 UTC m=+0.429878048,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-58.ec2.internal,}"
Sep 11 16:29:40.800556 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.800543    2545 policy_none.go:47] "Start"
Sep 11 16:29:40.806400 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.806383    2545 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g2t6v"
Sep 11 16:29:40.808122 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.808068    2545 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-58.ec2.internal.18d451273448b6af  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-58.ec2.internal,UID:ip-10-0-135-58.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-58.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-58.ec2.internal,},FirstTimestamp:2026-09-11 16:29:40.795684527 +0000 UTC m=+0.429899046,LastTimestamp:2026-09-11 16:29:40.795684527 +0000 UTC m=+0.429899046,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-58.ec2.internal,}"
Sep 11 16:29:40.844475 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.844462    2545 manager.go:337] "Starting Device Plugin manager"
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.844496    2545 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.844504    2545 server.go:87] "Starting device plugin registration server"
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.844732    2545 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.844898    2545 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.844997    2545 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.845068    2545 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.845077    2545 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.845567    2545 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.845606    2545 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-58.ec2.internal\" not found"
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.848192    2545 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.849492    2545 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.849551    2545 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.849579    2545 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.849622    2545 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 16:29:40.859029 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.852154    2545 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:29:40.945785 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.945732    2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:29:40.946495 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.946482    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:29:40.946545 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.946509    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:29:40.946545 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.946525    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:29:40.946545 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.946545    2545 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-58.ec2.internal"
Sep 11 16:29:40.949708 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.949687    2545 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-58.ec2.internal"]
Sep 11 16:29:40.949759 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.949754    2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:29:40.951154 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.951140    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:29:40.951212 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.951168    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:29:40.951212 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.951177    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:29:40.953426 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.953414    2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:29:40.953577 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.953561    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:40.953643 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.953597    2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:29:40.954538 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.954521    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:29:40.954538 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.954534    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:29:40.954718 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.954547    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:29:40.954718 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.954559    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:29:40.954718 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.954558    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:29:40.954718 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.954591    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:29:40.954718 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.954641    2545 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-58.ec2.internal"
Sep 11 16:29:40.954718 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.954659    2545 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-58.ec2.internal\": node \"ip-10-0-135-58.ec2.internal\" not found"
Sep 11 16:29:40.956794 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.956777    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:40.956887 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.956811    2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:29:40.957577 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.957557    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:29:40.957649 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.957590    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:29:40.957649 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.957600    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:29:40.977389 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.977369    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/59c79f2c70829d40ee96e19328eda4a1-config\") pod \"kube-apiserver-proxy-ip-10-0-135-58.ec2.internal\" (UID: \"59c79f2c70829d40ee96e19328eda4a1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:40.977500 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.977401    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c807854f91db83feccbd9633f71fa537-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal\" (UID: \"c807854f91db83feccbd9633f71fa537\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:40.977569 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.977552    2545 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-58.ec2.internal\" not found" node="ip-10-0-135-58.ec2.internal"
Sep 11 16:29:40.977871 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:40.977425    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c807854f91db83feccbd9633f71fa537-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal\" (UID: \"c807854f91db83feccbd9633f71fa537\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:40.981984 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:40.981970    2545 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-58.ec2.internal\" not found" node="ip-10-0-135-58.ec2.internal"
Sep 11 16:29:41.043170 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:41.043153    2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-58.ec2.internal\" not found"
Sep 11 16:29:41.078805 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.078785    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/59c79f2c70829d40ee96e19328eda4a1-config\") pod \"kube-apiserver-proxy-ip-10-0-135-58.ec2.internal\" (UID: \"59c79f2c70829d40ee96e19328eda4a1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:41.078896 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.078807    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c807854f91db83feccbd9633f71fa537-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal\" (UID: \"c807854f91db83feccbd9633f71fa537\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:41.078896 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.078841    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c807854f91db83feccbd9633f71fa537-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal\" (UID: \"c807854f91db83feccbd9633f71fa537\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:41.078896 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.078874    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/59c79f2c70829d40ee96e19328eda4a1-config\") pod \"kube-apiserver-proxy-ip-10-0-135-58.ec2.internal\" (UID: \"59c79f2c70829d40ee96e19328eda4a1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:41.079009 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.078890    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c807854f91db83feccbd9633f71fa537-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal\" (UID: \"c807854f91db83feccbd9633f71fa537\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:41.079009 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.078880    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c807854f91db83feccbd9633f71fa537-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal\" (UID: \"c807854f91db83feccbd9633f71fa537\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:41.144122 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:41.144100    2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-58.ec2.internal\" not found"
Sep 11 16:29:41.244961 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:41.244899    2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-58.ec2.internal\" not found"
Sep 11 16:29:41.281456 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.281434    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:41.287024 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.286613    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:41.345791 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:41.345766    2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-58.ec2.internal\" not found"
Sep 11 16:29:41.446225 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:41.446206    2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-58.ec2.internal\" not found"
Sep 11 16:29:41.546770 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:41.546717    2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-58.ec2.internal\" not found"
Sep 11 16:29:41.647195 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:41.647170    2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-58.ec2.internal\" not found"
Sep 11 16:29:41.707657 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.707636    2545 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 16:29:41.708014 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.707775    2545 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 16:29:41.747934 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:41.747912    2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-58.ec2.internal\" not found"
Sep 11 16:29:41.803685 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.803618    2545 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:29:41.810399 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.810359    2545 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 16:24:41 +0000 UTC" deadline="2028-06-25 01:36:46.8590128 +0000 UTC"
Sep 11 16:29:41.810399 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.810385    2545 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15657h7m5.048630794s"
Sep 11 16:29:41.842522 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.842504    2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:29:41.843737 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.843722    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:29:41.843839 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.843752    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:29:41.843839 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.843765    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:29:41.848077 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:41.848059    2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-58.ec2.internal\" not found"
Sep 11 16:29:41.898856 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.898835    2545 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:29:41.900485 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:41.900448    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c79f2c70829d40ee96e19328eda4a1.slice/crio-d39cd3df7b4b3e2bf7bb02fa2b7f25ac831d107d391715b3d179c14ffeb94e31 WatchSource:0}: Error finding container d39cd3df7b4b3e2bf7bb02fa2b7f25ac831d107d391715b3d179c14ffeb94e31: Status 404 returned error can't find the container with id d39cd3df7b4b3e2bf7bb02fa2b7f25ac831d107d391715b3d179c14ffeb94e31
Sep 11 16:29:41.901196 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:41.901177    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc807854f91db83feccbd9633f71fa537.slice/crio-067dbd6007a161206752452b767970190b9e16cccb07b3bc49ac2f8069034fd5 WatchSource:0}: Error finding container 067dbd6007a161206752452b767970190b9e16cccb07b3bc49ac2f8069034fd5: Status 404 returned error can't find the container with id 067dbd6007a161206752452b767970190b9e16cccb07b3bc49ac2f8069034fd5
Sep 11 16:29:41.906703 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.906689    2545 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 16:29:41.948338 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:41.948314    2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-58.ec2.internal\" not found"
Sep 11 16:29:41.974204 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.974187    2545 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:29:41.975625 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.975611    2545 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:41.987224 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.987210    2545 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 16:29:41.988636 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.988624    2545 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-58.ec2.internal"
Sep 11 16:29:41.998105 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:41.998088    2545 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 16:29:43.675804 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.675781    2545 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:29:43.835221 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.835192    2545 apiserver.go:52] "Watching apiserver"
Sep 11 16:29:43.841030 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.841009    2545 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 16:29:43.842411 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.842383    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-47mhr","openshift-dns/node-resolver-lxlpg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal","openshift-multus/multus-8bz87","openshift-multus/network-metrics-daemon-56kmc","kube-system/global-pull-secret-syncer-wfmld","kube-system/konnectivity-agent-tzdpv","kube-system/kube-apiserver-proxy-ip-10-0-135-58.ec2.internal","openshift-image-registry/node-ca-8vt9w","openshift-multus/multus-additional-cni-plugins-mbtf2","openshift-network-diagnostics/network-check-target-fhqvw","openshift-network-operator/iptables-alerter-x8x5h","openshift-ovn-kubernetes/ovnkube-node-9jtmq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"]
Sep 11 16:29:43.844922 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.844901    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tzdpv"
Sep 11 16:29:43.846979 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.846953    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 16:29:43.846979 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.846970    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 16:29:43.847150 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.846973    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lzvtt\""
Sep 11 16:29:43.847253 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.847237    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.849352 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.849331    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 16:29:43.849544 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.849530    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 16:29:43.849728 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.849713    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-224kn\""
Sep 11 16:29:43.850514 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.850209    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 16:29:43.850514 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.850220    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 16:29:43.850514 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.850320    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 16:29:43.850514 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.850425    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 16:29:43.850514 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.850471    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.852425 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.852398    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hvqvj\""
Sep 11 16:29:43.852522 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.852437    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 16:29:43.852681 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.852664    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.852775 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.852698    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 16:29:43.852912 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.852898    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 16:29:43.855012 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.854991    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5h4lq\""
Sep 11 16:29:43.855425 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.855403    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 16:29:43.855752 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.855568    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 16:29:43.855752 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.855413    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 16:29:43.855752 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.855751    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 16:29:43.858254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.856088    2545 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 16:29:43.858254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.856245    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:43.858254 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:43.856325    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:29:43.858501 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.858477    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:43.858654 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:43.858561    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:29:43.861899 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.861879    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8vt9w"
Sep 11 16:29:43.865070 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.864700    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f8jpd\""
Sep 11 16:29:43.865070 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.864721    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 16:29:43.865070 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.864752    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 16:29:43.865070 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.864923    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 16:29:43.866197 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.866176    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868013    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-run-k8s-cni-cncf-io\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868045    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-multus-conf-dir\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868071    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgdfh\" (UniqueName: \"kubernetes.io/projected/17b64265-fb83-45fe-820a-c5c6e0dffad7-kube-api-access-dgdfh\") pod \"network-metrics-daemon-56kmc\" (UID: \"17b64265-fb83-45fe-820a-c5c6e0dffad7\") " pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868087    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868109    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/84351f8e-eeb4-43fc-9e43-465096bcf573-agent-certs\") pod \"konnectivity-agent-tzdpv\" (UID: \"84351f8e-eeb4-43fc-9e43-465096bcf573\") " pod="kube-system/konnectivity-agent-tzdpv"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868132    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-etc-openvswitch\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868152    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-run-ovn\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868183    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-env-overrides\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868207    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/504b60b6-7663-4c59-92c8-411ef5c25201-multus-daemon-config\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868228    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-kubelet\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868250    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-node-log\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868273    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-run-ovn-kubernetes\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868297    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hw42\" (UniqueName: \"kubernetes.io/projected/9254b8fa-9575-4aab-9e50-d1633a74d081-kube-api-access-4hw42\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868319    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-run-netns\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868339    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf6faf2b-abcc-4abe-bf4f-d7496123108c-dbus\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:43.868882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868357    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868371    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868388    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-system-cni-dir\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868410    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-var-lib-cni-bin\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868428    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pndm\" (UniqueName: \"kubernetes.io/projected/504b60b6-7663-4c59-92c8-411ef5c25201-kube-api-access-4pndm\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868444    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-var-lib-openvswitch\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868465    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nptb\" (UniqueName: \"kubernetes.io/projected/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-kube-api-access-5nptb\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868486    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-device-dir\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868501    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-var-lib-kubelet\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868553    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-97wtp\""
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868567    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-run-multus-certs\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868582    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs\") pod \"network-metrics-daemon-56kmc\" (UID: \"17b64265-fb83-45fe-820a-c5c6e0dffad7\") " pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868595    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-ovnkube-script-lib\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868611    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-registration-dir\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868635    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-etc-selinux\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868656    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-sys-fs\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868705    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-cnibin\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.869637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868724    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/504b60b6-7663-4c59-92c8-411ef5c25201-cni-binary-copy\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868741    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-multus-socket-dir-parent\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868755    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf6faf2b-abcc-4abe-bf4f-d7496123108c-kubelet-config\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868772    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-run-openvswitch\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868785    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868793    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-log-socket\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868807    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-multus-cni-dir\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868842    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-var-lib-cni-multus\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868883    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-hostroot\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868907    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/84351f8e-eeb4-43fc-9e43-465096bcf573-konnectivity-ca\") pod \"konnectivity-agent-tzdpv\" (UID: \"84351f8e-eeb4-43fc-9e43-465096bcf573\") " pod="kube-system/konnectivity-agent-tzdpv"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868932    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-systemd-units\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868950    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868954    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-slash\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868977    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-cni-bin\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.868999    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-cni-netd\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.869023    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-ovnkube-config\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.869046    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-etc-kubernetes\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.869071    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-socket-dir\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.870439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.869094    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-run-netns\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.871280 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.869290    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-run-systemd\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.871280 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.869324    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-ovn-node-metrics-cert\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.871280 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.869560    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lxlpg"
Sep 11 16:29:43.871600 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.871583    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 16:29:43.872370 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.871977    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9cddb\""
Sep 11 16:29:43.872370 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.872163    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 16:29:43.872842 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.872796    2545 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 16:29:43.873104 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.873088    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:43.874776 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.874757    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g98cr\""
Sep 11 16:29:43.874920 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.874901    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 16:29:43.875016 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.875008    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 16:29:43.875677 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.875662    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:43.875758 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:43.875719    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:29:43.877633 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.877609    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x8x5h"
Sep 11 16:29:43.879595 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.879576    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dq98t\""
Sep 11 16:29:43.879687 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.879622    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 16:29:43.879687 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.879639    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:29:43.879783 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.879581    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 16:29:43.897575 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.897555    2545 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jjxjf"
Sep 11 16:29:43.906461 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.906445    2545 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jjxjf"
Sep 11 16:29:43.936669 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.936627    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-58.ec2.internal" event={"ID":"59c79f2c70829d40ee96e19328eda4a1","Type":"ContainerStarted","Data":"d39cd3df7b4b3e2bf7bb02fa2b7f25ac831d107d391715b3d179c14ffeb94e31"}
Sep 11 16:29:43.937439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.937413    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal" event={"ID":"c807854f91db83feccbd9633f71fa537","Type":"ContainerStarted","Data":"067dbd6007a161206752452b767970190b9e16cccb07b3bc49ac2f8069034fd5"}
Sep 11 16:29:43.958052 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.958035    2545 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 16:29:43.969878 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.969842    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-etc-kubernetes\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.969974 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.969898    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh9h2\" (UniqueName: \"kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2\") pod \"network-check-target-fhqvw\" (UID: \"eaffdb73-faca-4808-8b86-0783dffb6197\") " pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:43.969974 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.969920    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbk28\" (UniqueName: \"kubernetes.io/projected/5415fcf3-0701-4c3a-8582-6a614a28ab4a-kube-api-access-mbk28\") pod \"node-ca-8vt9w\" (UID: \"5415fcf3-0701-4c3a-8582-6a614a28ab4a\") " pod="openshift-image-registry/node-ca-8vt9w"
Sep 11 16:29:43.969974 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.969936    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-sysctl-d\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.969974 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.969959    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-systemd\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.970207 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.969978    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-run-netns\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.970207 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.969975    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-etc-kubernetes\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.970207 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.969994    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:43.970207 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970035    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/998c5c97-0ee4-467e-8915-4861c2d0ee51-iptables-alerter-script\") pod \"iptables-alerter-x8x5h\" (UID: \"998c5c97-0ee4-467e-8915-4861c2d0ee51\") " pod="openshift-network-operator/iptables-alerter-x8x5h"
Sep 11 16:29:43.970207 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:43.970101    2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:29:43.970207 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970154    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-run-netns\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.970207 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:43.970167    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret podName:cf6faf2b-abcc-4abe-bf4f-d7496123108c nodeName:}" failed. No retries permitted until 2026-09-11 16:29:44.470145955 +0000 UTC m=+3.023315880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret") pod "global-pull-secret-syncer-wfmld" (UID: "cf6faf2b-abcc-4abe-bf4f-d7496123108c") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:29:43.970207 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970156    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9350ed8f-0176-49c6-b2c7-92ee69b2c79a-hosts-file\") pod \"node-resolver-lxlpg\" (UID: \"9350ed8f-0176-49c6-b2c7-92ee69b2c79a\") " pod="openshift-dns/node-resolver-lxlpg"
Sep 11 16:29:43.970207 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970187    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/189a1a33-c42e-49b2-a086-c7317050492a-cni-binary-copy\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:43.970207 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970205    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/189a1a33-c42e-49b2-a086-c7317050492a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970227    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/189a1a33-c42e-49b2-a086-c7317050492a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970255    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/189a1a33-c42e-49b2-a086-c7317050492a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970275    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/84351f8e-eeb4-43fc-9e43-465096bcf573-agent-certs\") pod \"konnectivity-agent-tzdpv\" (UID: \"84351f8e-eeb4-43fc-9e43-465096bcf573\") " pod="kube-system/konnectivity-agent-tzdpv"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970295    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-run-ovn\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970309    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-env-overrides\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970322    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/504b60b6-7663-4c59-92c8-411ef5c25201-multus-daemon-config\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970337    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-var-lib-kubelet\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970352    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-run-ovn-kubernetes\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970371    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hw42\" (UniqueName: \"kubernetes.io/projected/9254b8fa-9575-4aab-9e50-d1633a74d081-kube-api-access-4hw42\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970386    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-run-netns\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970406    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5415fcf3-0701-4c3a-8582-6a614a28ab4a-serviceca\") pod \"node-ca-8vt9w\" (UID: \"5415fcf3-0701-4c3a-8582-6a614a28ab4a\") " pod="openshift-image-registry/node-ca-8vt9w"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970419    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-sys\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970438    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-tuned\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970452    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngqdg\" (UniqueName: \"kubernetes.io/projected/9350ed8f-0176-49c6-b2c7-92ee69b2c79a-kube-api-access-ngqdg\") pod \"node-resolver-lxlpg\" (UID: \"9350ed8f-0176-49c6-b2c7-92ee69b2c79a\") " pod="openshift-dns/node-resolver-lxlpg"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970473    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/189a1a33-c42e-49b2-a086-c7317050492a-system-cni-dir\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970491    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.970670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970510    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-system-cni-dir\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970538    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5415fcf3-0701-4c3a-8582-6a614a28ab4a-host\") pod \"node-ca-8vt9w\" (UID: \"5415fcf3-0701-4c3a-8582-6a614a28ab4a\") " pod="openshift-image-registry/node-ca-8vt9w"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970559    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-modprobe-d\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970578    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf6faf2b-abcc-4abe-bf4f-d7496123108c-dbus\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970601    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-var-lib-kubelet\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970617    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-ovnkube-script-lib\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970640    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-registration-dir\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970676    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-etc-selinux\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970694    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/504b60b6-7663-4c59-92c8-411ef5c25201-cni-binary-copy\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970719    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-multus-socket-dir-parent\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970745    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf6faf2b-abcc-4abe-bf4f-d7496123108c-kubelet-config\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970741    2545 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970770    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-run-openvswitch\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970789    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-log-socket\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970810    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-multus-cni-dir\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970828    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-var-lib-cni-multus\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970843    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/84351f8e-eeb4-43fc-9e43-465096bcf573-konnectivity-ca\") pod \"konnectivity-agent-tzdpv\" (UID: \"84351f8e-eeb4-43fc-9e43-465096bcf573\") " pod="kube-system/konnectivity-agent-tzdpv"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970877    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-systemd-units\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.971394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970900    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-slash\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970915    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-sysconfig\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970929    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-sysctl-conf\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970941    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-host\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970953    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-run-ovn\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970960    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-socket-dir\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.970996    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-run-systemd\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971016    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-ovn-node-metrics-cert\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971037    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-run-k8s-cni-cncf-io\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971061    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-socket-dir\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971060    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-multus-conf-dir\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971091    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgdfh\" (UniqueName: \"kubernetes.io/projected/17b64265-fb83-45fe-820a-c5c6e0dffad7-kube-api-access-dgdfh\") pod \"network-metrics-daemon-56kmc\" (UID: \"17b64265-fb83-45fe-820a-c5c6e0dffad7\") " pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971091    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-run-openvswitch\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971156    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-log-socket\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971302    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-multus-cni-dir\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971340    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-var-lib-cni-multus\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971423    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf6faf2b-abcc-4abe-bf4f-d7496123108c-dbus\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:43.972254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971487    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-env-overrides\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971512    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-var-lib-kubelet\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971613    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-multus-socket-dir-parent\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971631    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-registration-dir\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971685    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf6faf2b-abcc-4abe-bf4f-d7496123108c-kubelet-config\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971690    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-systemd-units\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971695    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-etc-selinux\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971721    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-multus-conf-dir\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971115    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlsnq\" (UniqueName: \"kubernetes.io/projected/998c5c97-0ee4-467e-8915-4861c2d0ee51-kube-api-access-tlsnq\") pod \"iptables-alerter-x8x5h\" (UID: \"998c5c97-0ee4-467e-8915-4861c2d0ee51\") " pod="openshift-network-operator/iptables-alerter-x8x5h"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971759    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-etc-openvswitch\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971755    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-slash\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971825    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-run\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971851    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8td8m\" (UniqueName: \"kubernetes.io/projected/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-kube-api-access-8td8m\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971898    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9350ed8f-0176-49c6-b2c7-92ee69b2c79a-tmp-dir\") pod \"node-resolver-lxlpg\" (UID: \"9350ed8f-0176-49c6-b2c7-92ee69b2c79a\") " pod="openshift-dns/node-resolver-lxlpg"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971923    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-kubelet\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971948    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-node-log\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971971    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/998c5c97-0ee4-467e-8915-4861c2d0ee51-host-slash\") pod \"iptables-alerter-x8x5h\" (UID: \"998c5c97-0ee4-467e-8915-4861c2d0ee51\") " pod="openshift-network-operator/iptables-alerter-x8x5h"
Sep 11 16:29:43.973087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.971995    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972020    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-var-lib-cni-bin\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972035    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/504b60b6-7663-4c59-92c8-411ef5c25201-multus-daemon-config\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972049    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pndm\" (UniqueName: \"kubernetes.io/projected/504b60b6-7663-4c59-92c8-411ef5c25201-kube-api-access-4pndm\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972073    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-kubernetes\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972126    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972158    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-run-netns\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972162    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-run-systemd\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972193    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-var-lib-cni-bin\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972210    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-lib-modules\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972208    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-run-k8s-cni-cncf-io\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972230    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-run-ovn-kubernetes\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972268    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/504b60b6-7663-4c59-92c8-411ef5c25201-cni-binary-copy\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972338    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-tmp\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972348    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-node-log\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972370    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-var-lib-openvswitch\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972371    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.973903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972385    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-kubelet\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972417    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nptb\" (UniqueName: \"kubernetes.io/projected/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-kube-api-access-5nptb\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972423    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-etc-openvswitch\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972424    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-var-lib-openvswitch\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972443    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-device-dir\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972494    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-device-dir\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972495    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-system-cni-dir\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972525    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-run-multus-certs\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972633    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs\") pod \"network-metrics-daemon-56kmc\" (UID: \"17b64265-fb83-45fe-820a-c5c6e0dffad7\") " pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972641    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-host-run-multus-certs\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972665    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs5m8\" (UniqueName: \"kubernetes.io/projected/189a1a33-c42e-49b2-a086-c7317050492a-kube-api-access-hs5m8\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:43.972709    2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972759    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-sys-fs\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:43.972772    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs podName:17b64265-fb83-45fe-820a-c5c6e0dffad7 nodeName:}" failed. No retries permitted until 2026-09-11 16:29:44.472755428 +0000 UTC m=+3.025925341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs") pod "network-metrics-daemon-56kmc" (UID: "17b64265-fb83-45fe-820a-c5c6e0dffad7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972836    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9254b8fa-9575-4aab-9e50-d1633a74d081-sys-fs\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972839    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-cnibin\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972931    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-cnibin\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.974662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.972988    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/189a1a33-c42e-49b2-a086-c7317050492a-cnibin\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:43.981616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.973017    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-hostroot\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.981616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.973066    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-cni-bin\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.981616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.973094    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-cni-netd\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.981616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.973105    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/84351f8e-eeb4-43fc-9e43-465096bcf573-konnectivity-ca\") pod \"konnectivity-agent-tzdpv\" (UID: \"84351f8e-eeb4-43fc-9e43-465096bcf573\") " pod="kube-system/konnectivity-agent-tzdpv"
Sep 11 16:29:43.981616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.973136    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-ovnkube-config\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.981616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.973162    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/504b60b6-7663-4c59-92c8-411ef5c25201-hostroot\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.981616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.973209    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-cni-bin\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.981616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.973250    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-host-cni-netd\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.981616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.973299    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-ovnkube-script-lib\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.981616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.973643    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-ovnkube-config\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.981616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.975358    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/84351f8e-eeb4-43fc-9e43-465096bcf573-agent-certs\") pod \"konnectivity-agent-tzdpv\" (UID: \"84351f8e-eeb4-43fc-9e43-465096bcf573\") " pod="kube-system/konnectivity-agent-tzdpv"
Sep 11 16:29:43.981616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.975797    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-ovn-node-metrics-cert\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:43.982512 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.982493    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgdfh\" (UniqueName: \"kubernetes.io/projected/17b64265-fb83-45fe-820a-c5c6e0dffad7-kube-api-access-dgdfh\") pod \"network-metrics-daemon-56kmc\" (UID: \"17b64265-fb83-45fe-820a-c5c6e0dffad7\") " pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:43.983171 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.983154    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pndm\" (UniqueName: \"kubernetes.io/projected/504b60b6-7663-4c59-92c8-411ef5c25201-kube-api-access-4pndm\") pod \"multus-8bz87\" (UID: \"504b60b6-7663-4c59-92c8-411ef5c25201\") " pod="openshift-multus/multus-8bz87"
Sep 11 16:29:43.983283 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.983267    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hw42\" (UniqueName: \"kubernetes.io/projected/9254b8fa-9575-4aab-9e50-d1633a74d081-kube-api-access-4hw42\") pod \"aws-ebs-csi-driver-node-gbswp\" (UID: \"9254b8fa-9575-4aab-9e50-d1633a74d081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:43.984311 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:43.984296    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nptb\" (UniqueName: \"kubernetes.io/projected/cb20fb2b-6b4b-491e-894a-7a9cf6c3832d-kube-api-access-5nptb\") pod \"ovnkube-node-9jtmq\" (UID: \"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:44.074202 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074170    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-var-lib-kubelet\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.074202 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074212    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5415fcf3-0701-4c3a-8582-6a614a28ab4a-serviceca\") pod \"node-ca-8vt9w\" (UID: \"5415fcf3-0701-4c3a-8582-6a614a28ab4a\") " pod="openshift-image-registry/node-ca-8vt9w"
Sep 11 16:29:44.074202 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074237    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-sys\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.074499 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074258    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-tuned\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.074499 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074282    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngqdg\" (UniqueName: \"kubernetes.io/projected/9350ed8f-0176-49c6-b2c7-92ee69b2c79a-kube-api-access-ngqdg\") pod \"node-resolver-lxlpg\" (UID: \"9350ed8f-0176-49c6-b2c7-92ee69b2c79a\") " pod="openshift-dns/node-resolver-lxlpg"
Sep 11 16:29:44.074499 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074305    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/189a1a33-c42e-49b2-a086-c7317050492a-system-cni-dir\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.074499 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074327    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5415fcf3-0701-4c3a-8582-6a614a28ab4a-host\") pod \"node-ca-8vt9w\" (UID: \"5415fcf3-0701-4c3a-8582-6a614a28ab4a\") " pod="openshift-image-registry/node-ca-8vt9w"
Sep 11 16:29:44.074499 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074348    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-modprobe-d\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.074499 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074384    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-sysconfig\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.074499 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074407    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-sysctl-conf\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.074499 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074442    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-host\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.074499 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074474    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlsnq\" (UniqueName: \"kubernetes.io/projected/998c5c97-0ee4-467e-8915-4861c2d0ee51-kube-api-access-tlsnq\") pod \"iptables-alerter-x8x5h\" (UID: \"998c5c97-0ee4-467e-8915-4861c2d0ee51\") " pod="openshift-network-operator/iptables-alerter-x8x5h"
Sep 11 16:29:44.074499 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074498    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-run\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074521    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8td8m\" (UniqueName: \"kubernetes.io/projected/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-kube-api-access-8td8m\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074546    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9350ed8f-0176-49c6-b2c7-92ee69b2c79a-tmp-dir\") pod \"node-resolver-lxlpg\" (UID: \"9350ed8f-0176-49c6-b2c7-92ee69b2c79a\") " pod="openshift-dns/node-resolver-lxlpg"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074570    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/998c5c97-0ee4-467e-8915-4861c2d0ee51-host-slash\") pod \"iptables-alerter-x8x5h\" (UID: \"998c5c97-0ee4-467e-8915-4861c2d0ee51\") " pod="openshift-network-operator/iptables-alerter-x8x5h"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074600    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-kubernetes\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074624    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-lib-modules\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074677    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-tmp\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074723    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs5m8\" (UniqueName: \"kubernetes.io/projected/189a1a33-c42e-49b2-a086-c7317050492a-kube-api-access-hs5m8\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074754    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/189a1a33-c42e-49b2-a086-c7317050492a-cnibin\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074786    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh9h2\" (UniqueName: \"kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2\") pod \"network-check-target-fhqvw\" (UID: \"eaffdb73-faca-4808-8b86-0783dffb6197\") " pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074811    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbk28\" (UniqueName: \"kubernetes.io/projected/5415fcf3-0701-4c3a-8582-6a614a28ab4a-kube-api-access-mbk28\") pod \"node-ca-8vt9w\" (UID: \"5415fcf3-0701-4c3a-8582-6a614a28ab4a\") " pod="openshift-image-registry/node-ca-8vt9w"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074837    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-sysctl-d\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074881    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-systemd\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074921    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/998c5c97-0ee4-467e-8915-4861c2d0ee51-iptables-alerter-script\") pod \"iptables-alerter-x8x5h\" (UID: \"998c5c97-0ee4-467e-8915-4861c2d0ee51\") " pod="openshift-network-operator/iptables-alerter-x8x5h"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074948    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9350ed8f-0176-49c6-b2c7-92ee69b2c79a-hosts-file\") pod \"node-resolver-lxlpg\" (UID: \"9350ed8f-0176-49c6-b2c7-92ee69b2c79a\") " pod="openshift-dns/node-resolver-lxlpg"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074972    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/189a1a33-c42e-49b2-a086-c7317050492a-cni-binary-copy\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.074997    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/189a1a33-c42e-49b2-a086-c7317050492a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.075135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.075023    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/189a1a33-c42e-49b2-a086-c7317050492a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.076116 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.075051    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/189a1a33-c42e-49b2-a086-c7317050492a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.076116 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.075684    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/189a1a33-c42e-49b2-a086-c7317050492a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.076116 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.075769    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-var-lib-kubelet\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.076274 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.076174    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5415fcf3-0701-4c3a-8582-6a614a28ab4a-serviceca\") pod \"node-ca-8vt9w\" (UID: \"5415fcf3-0701-4c3a-8582-6a614a28ab4a\") " pod="openshift-image-registry/node-ca-8vt9w"
Sep 11 16:29:44.076274 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.076242    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-sys\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.076659 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.076593    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-kubernetes\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.076659 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.076657    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-host\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.076813 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.076687    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-lib-modules\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.076892 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.076855    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/189a1a33-c42e-49b2-a086-c7317050492a-system-cni-dir\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.076955 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.076919    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5415fcf3-0701-4c3a-8582-6a614a28ab4a-host\") pod \"node-ca-8vt9w\" (UID: \"5415fcf3-0701-4c3a-8582-6a614a28ab4a\") " pod="openshift-image-registry/node-ca-8vt9w"
Sep 11 16:29:44.077024 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.077002    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-modprobe-d\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.077121 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.077046    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-run\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.077121 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.077089    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-sysconfig\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.077200 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.077165    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-sysctl-d\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.077235 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.077213    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9350ed8f-0176-49c6-b2c7-92ee69b2c79a-hosts-file\") pod \"node-resolver-lxlpg\" (UID: \"9350ed8f-0176-49c6-b2c7-92ee69b2c79a\") " pod="openshift-dns/node-resolver-lxlpg"
Sep 11 16:29:44.077281 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.077259    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-systemd\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.077574 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.077552    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9350ed8f-0176-49c6-b2c7-92ee69b2c79a-tmp-dir\") pod \"node-resolver-lxlpg\" (UID: \"9350ed8f-0176-49c6-b2c7-92ee69b2c79a\") " pod="openshift-dns/node-resolver-lxlpg"
Sep 11 16:29:44.077637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.077550    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/189a1a33-c42e-49b2-a086-c7317050492a-cnibin\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.077686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.077668    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/998c5c97-0ee4-467e-8915-4861c2d0ee51-host-slash\") pod \"iptables-alerter-x8x5h\" (UID: \"998c5c97-0ee4-467e-8915-4861c2d0ee51\") " pod="openshift-network-operator/iptables-alerter-x8x5h"
Sep 11 16:29:44.077754 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.077738    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/998c5c97-0ee4-467e-8915-4861c2d0ee51-iptables-alerter-script\") pod \"iptables-alerter-x8x5h\" (UID: \"998c5c97-0ee4-467e-8915-4861c2d0ee51\") " pod="openshift-network-operator/iptables-alerter-x8x5h"
Sep 11 16:29:44.077853 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.077830    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/189a1a33-c42e-49b2-a086-c7317050492a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.078012 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.077992    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-sysctl-conf\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.078156 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.078138    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/189a1a33-c42e-49b2-a086-c7317050492a-cni-binary-copy\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.078453 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.078431    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/189a1a33-c42e-49b2-a086-c7317050492a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.079198 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.079107    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-etc-tuned\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.079198 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.079159    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-tmp\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.088340 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.088319    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbk28\" (UniqueName: \"kubernetes.io/projected/5415fcf3-0701-4c3a-8582-6a614a28ab4a-kube-api-access-mbk28\") pod \"node-ca-8vt9w\" (UID: \"5415fcf3-0701-4c3a-8582-6a614a28ab4a\") " pod="openshift-image-registry/node-ca-8vt9w"
Sep 11 16:29:44.088441 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:44.088395    2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:29:44.088441 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:44.088409    2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:29:44.088441 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:44.088418    2545 projected.go:196] Error preparing data for projected volume kube-api-access-sh9h2 for pod openshift-network-diagnostics/network-check-target-fhqvw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:29:44.088599 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:44.088469    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2 podName:eaffdb73-faca-4808-8b86-0783dffb6197 nodeName:}" failed. No retries permitted until 2026-09-11 16:29:44.588457137 +0000 UTC m=+3.141627042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sh9h2" (UniqueName: "kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2") pod "network-check-target-fhqvw" (UID: "eaffdb73-faca-4808-8b86-0783dffb6197") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:29:44.092057 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.092027    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngqdg\" (UniqueName: \"kubernetes.io/projected/9350ed8f-0176-49c6-b2c7-92ee69b2c79a-kube-api-access-ngqdg\") pod \"node-resolver-lxlpg\" (UID: \"9350ed8f-0176-49c6-b2c7-92ee69b2c79a\") " pod="openshift-dns/node-resolver-lxlpg"
Sep 11 16:29:44.092384 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.092366    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8td8m\" (UniqueName: \"kubernetes.io/projected/a02fd770-5e3c-4f6e-a49e-2fd279a3d932-kube-api-access-8td8m\") pod \"tuned-47mhr\" (UID: \"a02fd770-5e3c-4f6e-a49e-2fd279a3d932\") " pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.092845 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.092824    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlsnq\" (UniqueName: \"kubernetes.io/projected/998c5c97-0ee4-467e-8915-4861c2d0ee51-kube-api-access-tlsnq\") pod \"iptables-alerter-x8x5h\" (UID: \"998c5c97-0ee4-467e-8915-4861c2d0ee51\") " pod="openshift-network-operator/iptables-alerter-x8x5h"
Sep 11 16:29:44.092927 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.092908    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs5m8\" (UniqueName: \"kubernetes.io/projected/189a1a33-c42e-49b2-a086-c7317050492a-kube-api-access-hs5m8\") pod \"multus-additional-cni-plugins-mbtf2\" (UID: \"189a1a33-c42e-49b2-a086-c7317050492a\") " pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.159484 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.159385    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tzdpv"
Sep 11 16:29:44.167061 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.167037    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:29:44.172336 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:44.172311    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84351f8e_eeb4_43fc_9e43_465096bcf573.slice/crio-d8983b17ef4549c6eb9f9adff5f822cc4f4f1bc598acbae4be0c2ccb03a3eae5 WatchSource:0}: Error finding container d8983b17ef4549c6eb9f9adff5f822cc4f4f1bc598acbae4be0c2ccb03a3eae5: Status 404 returned error can't find the container with id d8983b17ef4549c6eb9f9adff5f822cc4f4f1bc598acbae4be0c2ccb03a3eae5
Sep 11 16:29:44.177335 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:44.177305    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb20fb2b_6b4b_491e_894a_7a9cf6c3832d.slice/crio-da70f44c2c28129694a368d2bc03e86155534bedaf8d788181a2c898d9e605a8 WatchSource:0}: Error finding container da70f44c2c28129694a368d2bc03e86155534bedaf8d788181a2c898d9e605a8: Status 404 returned error can't find the container with id da70f44c2c28129694a368d2bc03e86155534bedaf8d788181a2c898d9e605a8
Sep 11 16:29:44.177429 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.177404    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp"
Sep 11 16:29:44.185438 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.184328    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8bz87"
Sep 11 16:29:44.188671 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:44.188650    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9254b8fa_9575_4aab_9e50_d1633a74d081.slice/crio-5dce24d2058b569611b0773af1515646291d5145ccd76ba5aa48feebd925315a WatchSource:0}: Error finding container 5dce24d2058b569611b0773af1515646291d5145ccd76ba5aa48feebd925315a: Status 404 returned error can't find the container with id 5dce24d2058b569611b0773af1515646291d5145ccd76ba5aa48feebd925315a
Sep 11 16:29:44.190538 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.190520    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8vt9w"
Sep 11 16:29:44.193493 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:44.193474    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod504b60b6_7663_4c59_92c8_411ef5c25201.slice/crio-84a9b00ae9891a8d8065957150e009d5f9e0d1ff0659c7c933a2087689a4c238 WatchSource:0}: Error finding container 84a9b00ae9891a8d8065957150e009d5f9e0d1ff0659c7c933a2087689a4c238: Status 404 returned error can't find the container with id 84a9b00ae9891a8d8065957150e009d5f9e0d1ff0659c7c933a2087689a4c238
Sep 11 16:29:44.197062 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.197046    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-47mhr"
Sep 11 16:29:44.203754 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.203558    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lxlpg"
Sep 11 16:29:44.206804 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:44.206785    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda02fd770_5e3c_4f6e_a49e_2fd279a3d932.slice/crio-e1a5cf23c3d990627b3bc9c98513c84ef7e07f0746d95d71e8cb0d1d6a14a505 WatchSource:0}: Error finding container e1a5cf23c3d990627b3bc9c98513c84ef7e07f0746d95d71e8cb0d1d6a14a505: Status 404 returned error can't find the container with id e1a5cf23c3d990627b3bc9c98513c84ef7e07f0746d95d71e8cb0d1d6a14a505
Sep 11 16:29:44.211526 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.211509    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mbtf2"
Sep 11 16:29:44.213283 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:44.213262    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9350ed8f_0176_49c6_b2c7_92ee69b2c79a.slice/crio-0d230417ce0cf4dc96962d5dd21782f96f7b07bb921ef9f50f7905f96a494fb2 WatchSource:0}: Error finding container 0d230417ce0cf4dc96962d5dd21782f96f7b07bb921ef9f50f7905f96a494fb2: Status 404 returned error can't find the container with id 0d230417ce0cf4dc96962d5dd21782f96f7b07bb921ef9f50f7905f96a494fb2
Sep 11 16:29:44.217925 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.217904    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x8x5h"
Sep 11 16:29:44.477268 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.477178    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs\") pod \"network-metrics-daemon-56kmc\" (UID: \"17b64265-fb83-45fe-820a-c5c6e0dffad7\") " pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:44.477268 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.477227    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:44.477478 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:44.477328    2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:29:44.477478 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:44.477360    2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:29:44.477478 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:44.477399    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret podName:cf6faf2b-abcc-4abe-bf4f-d7496123108c nodeName:}" failed. No retries permitted until 2026-09-11 16:29:45.47738002 +0000 UTC m=+4.030549933 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret") pod "global-pull-secret-syncer-wfmld" (UID: "cf6faf2b-abcc-4abe-bf4f-d7496123108c") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:29:44.477478 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:44.477421    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs podName:17b64265-fb83-45fe-820a-c5c6e0dffad7 nodeName:}" failed. No retries permitted until 2026-09-11 16:29:45.477406946 +0000 UTC m=+4.030576864 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs") pod "network-metrics-daemon-56kmc" (UID: "17b64265-fb83-45fe-820a-c5c6e0dffad7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:29:44.678898 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.678846    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh9h2\" (UniqueName: \"kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2\") pod \"network-check-target-fhqvw\" (UID: \"eaffdb73-faca-4808-8b86-0783dffb6197\") " pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:44.679066 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:44.679040    2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:29:44.679066 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:44.679065    2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:29:44.679183 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:44.679077    2545 projected.go:196] Error preparing data for projected volume kube-api-access-sh9h2 for pod openshift-network-diagnostics/network-check-target-fhqvw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:29:44.679183 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:44.679140    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2 podName:eaffdb73-faca-4808-8b86-0783dffb6197 nodeName:}" failed. No retries permitted until 2026-09-11 16:29:45.679120049 +0000 UTC m=+4.232289957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sh9h2" (UniqueName: "kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2") pod "network-check-target-fhqvw" (UID: "eaffdb73-faca-4808-8b86-0783dffb6197") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:29:44.909839 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.909734    2545 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 16:24:43 +0000 UTC" deadline="2028-03-07 02:17:29.093128177 +0000 UTC"
Sep 11 16:29:44.909839 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.909793    2545 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13017h47m44.183339247s"
Sep 11 16:29:44.940941 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.940908    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tzdpv" event={"ID":"84351f8e-eeb4-43fc-9e43-465096bcf573","Type":"ContainerStarted","Data":"d8983b17ef4549c6eb9f9adff5f822cc4f4f1bc598acbae4be0c2ccb03a3eae5"}
Sep 11 16:29:44.942254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.942222    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8vt9w" event={"ID":"5415fcf3-0701-4c3a-8582-6a614a28ab4a","Type":"ContainerStarted","Data":"37f1a71f9f43c44587fe9c1164d087c149a9acb4981f3a64a2f7e8a0798f6818"}
Sep 11 16:29:44.943436 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.943410    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lxlpg" event={"ID":"9350ed8f-0176-49c6-b2c7-92ee69b2c79a","Type":"ContainerStarted","Data":"0d230417ce0cf4dc96962d5dd21782f96f7b07bb921ef9f50f7905f96a494fb2"}
Sep 11 16:29:44.944690 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.944664    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-47mhr" event={"ID":"a02fd770-5e3c-4f6e-a49e-2fd279a3d932","Type":"ContainerStarted","Data":"e1a5cf23c3d990627b3bc9c98513c84ef7e07f0746d95d71e8cb0d1d6a14a505"}
Sep 11 16:29:44.946065 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.946034    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8bz87" event={"ID":"504b60b6-7663-4c59-92c8-411ef5c25201","Type":"ContainerStarted","Data":"84a9b00ae9891a8d8065957150e009d5f9e0d1ff0659c7c933a2087689a4c238"}
Sep 11 16:29:44.947437 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.947415    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp" event={"ID":"9254b8fa-9575-4aab-9e50-d1633a74d081","Type":"ContainerStarted","Data":"5dce24d2058b569611b0773af1515646291d5145ccd76ba5aa48feebd925315a"}
Sep 11 16:29:44.948687 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:44.948662    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq" event={"ID":"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d","Type":"ContainerStarted","Data":"da70f44c2c28129694a368d2bc03e86155534bedaf8d788181a2c898d9e605a8"}
Sep 11 16:29:45.275395 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:45.275366    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod189a1a33_c42e_49b2_a086_c7317050492a.slice/crio-5b369889c0f710944dcdb6a33057dd2fd1df91b00e54c766f5107acf6db02ed8 WatchSource:0}: Error finding container 5b369889c0f710944dcdb6a33057dd2fd1df91b00e54c766f5107acf6db02ed8: Status 404 returned error can't find the container with id 5b369889c0f710944dcdb6a33057dd2fd1df91b00e54c766f5107acf6db02ed8
Sep 11 16:29:45.278443 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:29:45.278422    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod998c5c97_0ee4_467e_8915_4861c2d0ee51.slice/crio-18ad0a294bfae9c323c2a15c1f4776cf2efbe1ded3c2f441aaa580aa37b82791 WatchSource:0}: Error finding container 18ad0a294bfae9c323c2a15c1f4776cf2efbe1ded3c2f441aaa580aa37b82791: Status 404 returned error can't find the container with id 18ad0a294bfae9c323c2a15c1f4776cf2efbe1ded3c2f441aaa580aa37b82791
Sep 11 16:29:45.486189 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:45.486156    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs\") pod \"network-metrics-daemon-56kmc\" (UID: \"17b64265-fb83-45fe-820a-c5c6e0dffad7\") " pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:45.486359 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:45.486224    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:45.486421 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:45.486369    2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:29:45.486468 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:45.486425    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret podName:cf6faf2b-abcc-4abe-bf4f-d7496123108c nodeName:}" failed. No retries permitted until 2026-09-11 16:29:47.486407727 +0000 UTC m=+6.039577639 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret") pod "global-pull-secret-syncer-wfmld" (UID: "cf6faf2b-abcc-4abe-bf4f-d7496123108c") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:29:45.486846 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:45.486810    2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:29:45.486986 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:45.486857    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs podName:17b64265-fb83-45fe-820a-c5c6e0dffad7 nodeName:}" failed. No retries permitted until 2026-09-11 16:29:47.486842733 +0000 UTC m=+6.040012640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs") pod "network-metrics-daemon-56kmc" (UID: "17b64265-fb83-45fe-820a-c5c6e0dffad7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:29:45.694329 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:45.693723    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh9h2\" (UniqueName: \"kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2\") pod \"network-check-target-fhqvw\" (UID: \"eaffdb73-faca-4808-8b86-0783dffb6197\") " pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:45.694329 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:45.693898    2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:29:45.694329 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:45.693922    2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:29:45.694329 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:45.693934    2545 projected.go:196] Error preparing data for projected volume kube-api-access-sh9h2 for pod openshift-network-diagnostics/network-check-target-fhqvw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:29:45.694329 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:45.693986    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2 podName:eaffdb73-faca-4808-8b86-0783dffb6197 nodeName:}" failed. No retries permitted until 2026-09-11 16:29:47.693968678 +0000 UTC m=+6.247138600 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sh9h2" (UniqueName: "kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2") pod "network-check-target-fhqvw" (UID: "eaffdb73-faca-4808-8b86-0783dffb6197") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:29:45.911307 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:45.910696    2545 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 16:24:43 +0000 UTC" deadline="2028-03-29 14:31:02.184619493 +0000 UTC"
Sep 11 16:29:45.911307 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:45.910727    2545 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13558h1m16.273897057s"
Sep 11 16:29:45.933343 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:45.933027    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:45.933343 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:45.933147    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:29:45.933343 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:45.933027    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:45.933343 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:45.933262    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:29:45.933727 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:45.933709    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:45.934437 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:45.933808    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:29:45.953179 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:45.953146    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x8x5h" event={"ID":"998c5c97-0ee4-467e-8915-4861c2d0ee51","Type":"ContainerStarted","Data":"18ad0a294bfae9c323c2a15c1f4776cf2efbe1ded3c2f441aaa580aa37b82791"}
Sep 11 16:29:45.957717 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:45.957682    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbtf2" event={"ID":"189a1a33-c42e-49b2-a086-c7317050492a","Type":"ContainerStarted","Data":"5b369889c0f710944dcdb6a33057dd2fd1df91b00e54c766f5107acf6db02ed8"}
Sep 11 16:29:45.974724 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:45.974630    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-58.ec2.internal" event={"ID":"59c79f2c70829d40ee96e19328eda4a1","Type":"ContainerStarted","Data":"ad1f79c6182bf653fb50fe106e58abd97e9b77e18447366d816d686dd5d520b3"}
Sep 11 16:29:46.986614 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:46.986134    2545 generic.go:356] "Generic (PLEG): container finished" podID="c807854f91db83feccbd9633f71fa537" containerID="d2ea5ed7f23fdd60415ce5ba5f634dcb508409967c686e5d871052f124960204" exitCode=0
Sep 11 16:29:46.986614 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:46.986600    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal" event={"ID":"c807854f91db83feccbd9633f71fa537","Type":"ContainerDied","Data":"d2ea5ed7f23fdd60415ce5ba5f634dcb508409967c686e5d871052f124960204"}
Sep 11 16:29:47.008166 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:47.008111    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-58.ec2.internal" podStartSLOduration=4.008093082 podStartE2EDuration="4.008093082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:29:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:29:46.000856337 +0000 UTC m=+4.554026285" watchObservedRunningTime="2026-09-11 16:29:47.008093082 +0000 UTC m=+5.561263009"
Sep 11 16:29:47.507281 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:47.507245    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs\") pod \"network-metrics-daemon-56kmc\" (UID: \"17b64265-fb83-45fe-820a-c5c6e0dffad7\") " pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:47.507450 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:47.507314    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:47.507450 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:47.507436    2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:29:47.507552 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:47.507494    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret podName:cf6faf2b-abcc-4abe-bf4f-d7496123108c nodeName:}" failed. No retries permitted until 2026-09-11 16:29:51.507477064 +0000 UTC m=+10.060646975 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret") pod "global-pull-secret-syncer-wfmld" (UID: "cf6faf2b-abcc-4abe-bf4f-d7496123108c") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:29:47.507901 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:47.507882    2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:29:47.507991 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:47.507936    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs podName:17b64265-fb83-45fe-820a-c5c6e0dffad7 nodeName:}" failed. No retries permitted until 2026-09-11 16:29:51.50792103 +0000 UTC m=+10.061090950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs") pod "network-metrics-daemon-56kmc" (UID: "17b64265-fb83-45fe-820a-c5c6e0dffad7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:29:47.709631 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:47.708999    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh9h2\" (UniqueName: \"kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2\") pod \"network-check-target-fhqvw\" (UID: \"eaffdb73-faca-4808-8b86-0783dffb6197\") " pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:47.709631 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:47.709172    2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:29:47.709631 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:47.709190    2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:29:47.709631 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:47.709202    2545 projected.go:196] Error preparing data for projected volume kube-api-access-sh9h2 for pod openshift-network-diagnostics/network-check-target-fhqvw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:29:47.709631 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:47.709258    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2 podName:eaffdb73-faca-4808-8b86-0783dffb6197 nodeName:}" failed. No retries permitted until 2026-09-11 16:29:51.709239966 +0000 UTC m=+10.262409884 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sh9h2" (UniqueName: "kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2") pod "network-check-target-fhqvw" (UID: "eaffdb73-faca-4808-8b86-0783dffb6197") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:29:47.932998 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:47.932923    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:47.933148 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:47.933023    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:29:47.933386 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:47.933294    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:47.933386 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:47.933364    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:29:47.933577 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:47.933403    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:47.933577 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:47.933445    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:29:47.992293 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:47.992256    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tzdpv" event={"ID":"84351f8e-eeb4-43fc-9e43-465096bcf573","Type":"ContainerStarted","Data":"5afb06a8b44fc5caee8ee41019ebf630f23c2b2813e7a72e515ae956ee023eb5"}
Sep 11 16:29:47.996684 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:47.996655    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal" event={"ID":"c807854f91db83feccbd9633f71fa537","Type":"ContainerStarted","Data":"5cfbe533a1f33816500068011a4e9547e15b8023f41a59fbc8e047a2d257d0a7"}
Sep 11 16:29:48.024140 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:48.024086    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tzdpv" podStartSLOduration=3.403003479 podStartE2EDuration="6.024067781s" totalImagesPullingTime="2.621064302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:29:48.009777859 +0000 UTC m=+6.562947806" watchObservedRunningTime="2026-09-11 16:29:48.024067781 +0000 UTC m=+6.577237708"
Sep 11 16:29:48.231148 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:48.231031    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tzdpv"
Sep 11 16:29:48.232035 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:48.232012    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tzdpv"
Sep 11 16:29:48.250333 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:48.249943    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-58.ec2.internal" podStartSLOduration=5.249924229 podStartE2EDuration="5.249924229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:29:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:29:48.024977998 +0000 UTC m=+6.578147938" watchObservedRunningTime="2026-09-11 16:29:48.249924229 +0000 UTC m=+6.803094158"
Sep 11 16:29:49.933054 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:49.933021    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:49.933419 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:49.933021    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:49.933419 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:49.933161    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:29:49.933419 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:49.933021    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:49.933419 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:49.933271    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:29:49.933419 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:49.933315    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:29:50.001931 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:50.001489    2545 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 16:29:51.537987 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:51.537947    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs\") pod \"network-metrics-daemon-56kmc\" (UID: \"17b64265-fb83-45fe-820a-c5c6e0dffad7\") " pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:51.538432 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:51.538018    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:51.538432 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:51.538157    2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:29:51.538432 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:51.538218    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret podName:cf6faf2b-abcc-4abe-bf4f-d7496123108c nodeName:}" failed. No retries permitted until 2026-09-11 16:29:59.538199879 +0000 UTC m=+18.091369787 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret") pod "global-pull-secret-syncer-wfmld" (UID: "cf6faf2b-abcc-4abe-bf4f-d7496123108c") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:29:51.538432 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:51.538290    2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:29:51.538432 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:51.538325    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs podName:17b64265-fb83-45fe-820a-c5c6e0dffad7 nodeName:}" failed. No retries permitted until 2026-09-11 16:29:59.538312988 +0000 UTC m=+18.091482895 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs") pod "network-metrics-daemon-56kmc" (UID: "17b64265-fb83-45fe-820a-c5c6e0dffad7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:29:51.739268 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:51.739229    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh9h2\" (UniqueName: \"kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2\") pod \"network-check-target-fhqvw\" (UID: \"eaffdb73-faca-4808-8b86-0783dffb6197\") " pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:51.739453 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:51.739420    2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:29:51.739453 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:51.739443    2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:29:51.739453 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:51.739453    2545 projected.go:196] Error preparing data for projected volume kube-api-access-sh9h2 for pod openshift-network-diagnostics/network-check-target-fhqvw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:29:51.739612 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:51.739502    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2 podName:eaffdb73-faca-4808-8b86-0783dffb6197 nodeName:}" failed. No retries permitted until 2026-09-11 16:29:59.739487045 +0000 UTC m=+18.292656969 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sh9h2" (UniqueName: "kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2") pod "network-check-target-fhqvw" (UID: "eaffdb73-faca-4808-8b86-0783dffb6197") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:29:51.932040 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:51.931967    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:51.932040 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:51.931998    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:51.932243 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:51.931977    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:51.932243 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:51.932101    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:29:51.932243 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:51.932178    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:29:51.932395 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:51.932259    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:29:53.932117 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:53.931655    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:53.932117 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:53.931777    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:29:53.932117 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:53.931934    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:53.932117 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:53.932026    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:29:53.932117 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:53.932073    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:53.932764 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:53.932158    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:29:55.932501 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:55.931972    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:55.932501 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:55.932006    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:55.932501 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:55.932105    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:55.932501 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:55.932120    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:29:55.932501 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:55.932192    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:29:55.932501 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:55.932307    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:29:57.930898 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:57.930846    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:57.930898 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:57.930885    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:57.931456 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:57.931003    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:29:57.931456 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:57.931057    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:57.931456 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:57.931170    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:29:57.931456 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:57.931259    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:29:59.599099 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:59.599067    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs\") pod \"network-metrics-daemon-56kmc\" (UID: \"17b64265-fb83-45fe-820a-c5c6e0dffad7\") " pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:59.599607 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:59.599121    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:59.599607 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:59.599223    2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:29:59.599607 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:59.599233    2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:29:59.599607 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:59.599283    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret podName:cf6faf2b-abcc-4abe-bf4f-d7496123108c nodeName:}" failed. No retries permitted until 2026-09-11 16:30:15.599265079 +0000 UTC m=+34.152435005 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret") pod "global-pull-secret-syncer-wfmld" (UID: "cf6faf2b-abcc-4abe-bf4f-d7496123108c") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:29:59.599607 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:59.599300    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs podName:17b64265-fb83-45fe-820a-c5c6e0dffad7 nodeName:}" failed. No retries permitted until 2026-09-11 16:30:15.599292141 +0000 UTC m=+34.152462050 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs") pod "network-metrics-daemon-56kmc" (UID: "17b64265-fb83-45fe-820a-c5c6e0dffad7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:29:59.800477 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:59.800440    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh9h2\" (UniqueName: \"kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2\") pod \"network-check-target-fhqvw\" (UID: \"eaffdb73-faca-4808-8b86-0783dffb6197\") " pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:59.800658 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:59.800591    2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:29:59.800658 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:59.800609    2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:29:59.800658 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:59.800619    2545 projected.go:196] Error preparing data for projected volume kube-api-access-sh9h2 for pod openshift-network-diagnostics/network-check-target-fhqvw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:29:59.800819 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:59.800663    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2 podName:eaffdb73-faca-4808-8b86-0783dffb6197 nodeName:}" failed. No retries permitted until 2026-09-11 16:30:15.800650618 +0000 UTC m=+34.353820523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sh9h2" (UniqueName: "kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2") pod "network-check-target-fhqvw" (UID: "eaffdb73-faca-4808-8b86-0783dffb6197") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:29:59.931995 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:59.931917    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:29:59.932154 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:59.931917    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:29:59.932154 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:59.932023    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:29:59.932154 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:29:59.931919    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:29:59.932313 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:59.932101    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:29:59.932313 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:29:59.932212    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:30:01.642247 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:01.642213    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tzdpv"
Sep 11 16:30:01.642646 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:01.642340    2545 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 16:30:01.642793 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:01.642771    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tzdpv"
Sep 11 16:30:01.931615 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:01.931540    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:30:01.931615 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:01.931582    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:30:01.931763 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:01.931642    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:30:01.931763 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:01.931720    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:30:01.931763 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:01.931743    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:30:01.931881 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:01.931794    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:30:03.026360 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.026142    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lxlpg" event={"ID":"9350ed8f-0176-49c6-b2c7-92ee69b2c79a","Type":"ContainerStarted","Data":"649c2d7fc025c56a2cedd10a0682e32bb9113d6d0bdde199a8407f818f1ae6f8"}
Sep 11 16:30:03.028505 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.028471    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-47mhr" event={"ID":"a02fd770-5e3c-4f6e-a49e-2fd279a3d932","Type":"ContainerStarted","Data":"45a0e2b8fd0fa72e57d697fbddb5c04ada2aee7f896e6a8cbb32952d51a03016"}
Sep 11 16:30:03.030646 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.030596    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8bz87" event={"ID":"504b60b6-7663-4c59-92c8-411ef5c25201","Type":"ContainerStarted","Data":"d93719e21c10b309d5fedc38b3d3cba5d98feb1fd6e76a75f877cf5d26980244"}
Sep 11 16:30:03.032789 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.032765    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp" event={"ID":"9254b8fa-9575-4aab-9e50-d1633a74d081","Type":"ContainerStarted","Data":"5312250e47766fc98f177ee72fce3431886e87d0c6f90ae7246337fa42972111"}
Sep 11 16:30:03.034513 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.034493    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq" event={"ID":"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d","Type":"ContainerStarted","Data":"cd3d2227187401798cd9030b092e179cbbf8aaf1e297b41e7518baa3a72bf994"}
Sep 11 16:30:03.036063 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.036042    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8vt9w" event={"ID":"5415fcf3-0701-4c3a-8582-6a614a28ab4a","Type":"ContainerStarted","Data":"605808126cc7211e6d1c953d7f51395465bbac8e02cac27f5d2e3ce27ed7e658"}
Sep 11 16:30:03.037887 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.037741    2545 generic.go:356] "Generic (PLEG): container finished" podID="189a1a33-c42e-49b2-a086-c7317050492a" containerID="7d5fbd5170167480707263d07b9828838986fe12c8ee421dde87a1c06215e971" exitCode=0
Sep 11 16:30:03.037887 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.037769    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbtf2" event={"ID":"189a1a33-c42e-49b2-a086-c7317050492a","Type":"ContainerDied","Data":"7d5fbd5170167480707263d07b9828838986fe12c8ee421dde87a1c06215e971"}
Sep 11 16:30:03.049188 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.049140    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lxlpg" podStartSLOduration=2.514900249 podStartE2EDuration="21.04912704s" totalImagesPullingTime="18.534226791s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:30:03.048682364 +0000 UTC m=+21.601852318" watchObservedRunningTime="2026-09-11 16:30:03.04912704 +0000 UTC m=+21.602296966"
Sep 11 16:30:03.097833 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.097641    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8bz87" podStartSLOduration=2.376409746 podStartE2EDuration="21.097624533s" totalImagesPullingTime="18.721214787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:30:03.080312633 +0000 UTC m=+21.633482592" watchObservedRunningTime="2026-09-11 16:30:03.097624533 +0000 UTC m=+21.650794438"
Sep 11 16:30:03.098121 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.098076    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8vt9w" podStartSLOduration=2.553136614 podStartE2EDuration="21.098065351s" totalImagesPullingTime="18.544928737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:30:03.097887058 +0000 UTC m=+21.651056981" watchObservedRunningTime="2026-09-11 16:30:03.098065351 +0000 UTC m=+21.651235341"
Sep 11 16:30:03.134840 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.134799    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-47mhr" podStartSLOduration=2.572171556 podStartE2EDuration="21.134785224s" totalImagesPullingTime="18.562613668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:30:03.134564165 +0000 UTC m=+21.687734111" watchObservedRunningTime="2026-09-11 16:30:03.134785224 +0000 UTC m=+21.687955150"
Sep 11 16:30:03.931144 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.931112    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:30:03.931393 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.931112    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:30:03.931393 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:03.931233    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:30:03.931393 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:03.931291    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:30:03.931551 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:03.931403    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:30:03.931551 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:03.931480    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:30:04.045063 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:04.045033    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq" event={"ID":"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d","Type":"ContainerStarted","Data":"3b122556c04521d37c335428158647e5f2feb2f655151d52fdeef4fad60a13ac"}
Sep 11 16:30:04.045624 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:04.045105    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq" event={"ID":"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d","Type":"ContainerStarted","Data":"f26694a5be675ed791f0de29886d8ddc1ffe29396791815b86a65142ee1445f8"}
Sep 11 16:30:04.045624 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:04.045153    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq" event={"ID":"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d","Type":"ContainerStarted","Data":"451644f62358d88dc0a30c6abc35c16d0e9dc6c2d8cd11c29669b90484bbfd8e"}
Sep 11 16:30:04.045624 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:04.045203    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq" event={"ID":"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d","Type":"ContainerStarted","Data":"04c2cf0a8969787a949226d8b27b8000718ab10586d66d07f2c350411a0f78ef"}
Sep 11 16:30:04.045624 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:04.045248    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq" event={"ID":"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d","Type":"ContainerStarted","Data":"32b946a40d2439d018254e1d008be3c76cccc276b2a97951e25fc2466525396d"}
Sep 11 16:30:04.228214 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:04.228193    2545 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 16:30:04.942480 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:04.942396    2545 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T16:30:04.228208791Z","UUID":"9450117a-6ee5-4a9e-aa6d-773417c0c91b","Handler":null,"Name":"","Endpoint":""}
Sep 11 16:30:04.943834 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:04.943806    2545 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 16:30:04.943834 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:04.943833    2545 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 16:30:05.049178 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:05.049146    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp" event={"ID":"9254b8fa-9575-4aab-9e50-d1633a74d081","Type":"ContainerStarted","Data":"674fc4781485fbe9f9763da7927d58a4235267b6227982ea13f49d44f2cade43"}
Sep 11 16:30:05.050799 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:05.050768    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x8x5h" event={"ID":"998c5c97-0ee4-467e-8915-4861c2d0ee51","Type":"ContainerStarted","Data":"5dc34bd19ab2023f651eff6417ea0e7d1f3ff4f679c35679c0099ad72bca6c96"}
Sep 11 16:30:05.931004 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:05.930971    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:30:05.931116 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:05.930972    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:30:05.931116 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:05.931087    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:30:05.931203 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:05.930987    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:30:05.931203 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:05.931155    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:30:05.931275 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:05.931223    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:30:06.055048 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:06.055016    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp" event={"ID":"9254b8fa-9575-4aab-9e50-d1633a74d081","Type":"ContainerStarted","Data":"8a61180648b831a845c321b629718eebb10e68a85b32b653a9d6104fdd0f9325"}
Sep 11 16:30:06.058507 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:06.058487    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq" event={"ID":"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d","Type":"ContainerStarted","Data":"0f6cfb852fda3111f1ae74f4c1c997908021a65b2efe1aeb60475f078d06e82c"}
Sep 11 16:30:06.077996 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:06.077957    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x8x5h" podStartSLOduration=6.582177619 podStartE2EDuration="24.077943261s" totalImagesPullingTime="17.495765642s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:30:05.087543969 +0000 UTC m=+23.640713907" watchObservedRunningTime="2026-09-11 16:30:06.077943261 +0000 UTC m=+24.631113188"
Sep 11 16:30:06.078149 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:06.078132    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gbswp" podStartSLOduration=2.900486252 podStartE2EDuration="24.07812803s" totalImagesPullingTime="21.177641778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:29:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:30:06.077890001 +0000 UTC m=+24.631059948" watchObservedRunningTime="2026-09-11 16:30:06.07812803 +0000 UTC m=+24.631297956"
Sep 11 16:30:07.932073 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:07.931809    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:30:07.932508 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:07.931809    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:30:07.932508 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:07.932143    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:30:07.932508 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:07.931818    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:30:07.932508 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:07.932237    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:30:07.932508 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:07.932304    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:30:09.069560 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:09.069521    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq" event={"ID":"cb20fb2b-6b4b-491e-894a-7a9cf6c3832d","Type":"ContainerStarted","Data":"d44201ddd09337fc269df1267bd292b23a1980b118bf1fa05b9dbd38005afa3f"}
Sep 11 16:30:09.070429 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:09.069762    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:30:09.070429 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:09.069826    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:30:09.071518 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:09.071489    2545 generic.go:356] "Generic (PLEG): container finished" podID="189a1a33-c42e-49b2-a086-c7317050492a" containerID="fa963c0cda88a1966ebc5454d1d9db04a47a3926071761fdd7d5e200a359e433" exitCode=0
Sep 11 16:30:09.071608 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:09.071544    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbtf2" event={"ID":"189a1a33-c42e-49b2-a086-c7317050492a","Type":"ContainerDied","Data":"fa963c0cda88a1966ebc5454d1d9db04a47a3926071761fdd7d5e200a359e433"}
Sep 11 16:30:09.086253 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:09.086234    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:30:09.104272 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:09.104232    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq" podStartSLOduration=8.510020309 podStartE2EDuration="27.104218917s" totalImagesPullingTime="18.594198608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:30:09.104120228 +0000 UTC m=+27.657290174" watchObservedRunningTime="2026-09-11 16:30:09.104218917 +0000 UTC m=+27.657388822"
Sep 11 16:30:09.931811 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:09.931640    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:30:09.931991 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:09.931640    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:30:09.931991 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:09.931934    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:30:09.932101 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:09.932007    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:30:09.932101 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:09.931640    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:30:09.932184 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:09.932098    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:30:10.074523 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:10.074432    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:30:10.090517 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:10.090493    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:30:10.544742 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:10.544710    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fhqvw"]
Sep 11 16:30:10.544917 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:10.544898    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:30:10.545038 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:10.545014    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:30:10.545346 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:10.545294    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wfmld"]
Sep 11 16:30:10.545444 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:10.545398    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:30:10.545530 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:10.545508    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:30:10.547263 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:10.547239    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-56kmc"]
Sep 11 16:30:10.547356 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:10.547343    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:30:10.547458 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:10.547435    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:30:11.078616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:11.078572    2545 generic.go:356] "Generic (PLEG): container finished" podID="189a1a33-c42e-49b2-a086-c7317050492a" containerID="2ffab576539b4611857f7009257c36bcf71efdd4fc586e5bca25350fbd79887c" exitCode=0
Sep 11 16:30:11.079011 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:11.078647    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbtf2" event={"ID":"189a1a33-c42e-49b2-a086-c7317050492a","Type":"ContainerDied","Data":"2ffab576539b4611857f7009257c36bcf71efdd4fc586e5bca25350fbd79887c"}
Sep 11 16:30:11.932283 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:11.932168    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:30:11.932283 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:11.932209    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:30:11.932461 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:11.932293    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:30:11.932461 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:11.932305    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:30:11.932461 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:11.932375    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:30:11.932461 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:11.932453    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:30:12.083926 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:12.083895    2545 generic.go:356] "Generic (PLEG): container finished" podID="189a1a33-c42e-49b2-a086-c7317050492a" containerID="a7b51487a8335df186bbd7a16bf02e4ccdba53bb0c3f0b6784bffd98dc440da1" exitCode=0
Sep 11 16:30:12.084758 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:12.084740    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbtf2" event={"ID":"189a1a33-c42e-49b2-a086-c7317050492a","Type":"ContainerDied","Data":"a7b51487a8335df186bbd7a16bf02e4ccdba53bb0c3f0b6784bffd98dc440da1"}
Sep 11 16:30:13.932094 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:13.931891    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:30:13.932507 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:13.931939    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:30:13.932507 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:13.932202    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wfmld" podUID="cf6faf2b-abcc-4abe-bf4f-d7496123108c"
Sep 11 16:30:13.932507 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:13.932294    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhqvw" podUID="eaffdb73-faca-4808-8b86-0783dffb6197"
Sep 11 16:30:13.932507 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:13.931947    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:30:13.932507 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:13.932394    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56kmc" podUID="17b64265-fb83-45fe-820a-c5c6e0dffad7"
Sep 11 16:30:15.611456 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.611373    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:30:15.611953 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.611462    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs\") pod \"network-metrics-daemon-56kmc\" (UID: \"17b64265-fb83-45fe-820a-c5c6e0dffad7\") " pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:30:15.611953 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:15.611553    2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:30:15.611953 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:15.611565    2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:30:15.611953 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:15.611625    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret podName:cf6faf2b-abcc-4abe-bf4f-d7496123108c nodeName:}" failed. No retries permitted until 2026-09-11 16:30:47.611609697 +0000 UTC m=+66.164779620 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret") pod "global-pull-secret-syncer-wfmld" (UID: "cf6faf2b-abcc-4abe-bf4f-d7496123108c") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:30:15.611953 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:15.611646    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs podName:17b64265-fb83-45fe-820a-c5c6e0dffad7 nodeName:}" failed. No retries permitted until 2026-09-11 16:30:47.611635831 +0000 UTC m=+66.164805736 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs") pod "network-metrics-daemon-56kmc" (UID: "17b64265-fb83-45fe-820a-c5c6e0dffad7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:30:15.649926 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.649898    2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-58.ec2.internal" event="NodeReady"
Sep 11 16:30:15.650045 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.650035    2545 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 16:30:15.698853 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.698819    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-m4w4b"]
Sep 11 16:30:15.722829 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.722791    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dpm7g"]
Sep 11 16:30:15.722992 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.722954    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:15.725955 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.725808    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 16:30:15.726306 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.726284    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 16:30:15.726418 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.726293    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-55n2r\""
Sep 11 16:30:15.738674 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.738655    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m4w4b"]
Sep 11 16:30:15.738776 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.738690    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dpm7g"]
Sep 11 16:30:15.738835 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.738773    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dpm7g"
Sep 11 16:30:15.743103 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.743087    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 16:30:15.743230 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.743138    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 16:30:15.743301 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.743212    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 16:30:15.743407 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.743388    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-t57kr\""
Sep 11 16:30:15.812692 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.812668    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb3167ad-e92f-4c98-8e23-f68f3f46e399-config-volume\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:15.812838 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.812705    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjl74\" (UniqueName: \"kubernetes.io/projected/cb3167ad-e92f-4c98-8e23-f68f3f46e399-kube-api-access-kjl74\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:15.812838 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.812792    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh9h2\" (UniqueName: \"kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2\") pod \"network-check-target-fhqvw\" (UID: \"eaffdb73-faca-4808-8b86-0783dffb6197\") " pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:30:15.812984 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:15.812850    2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:30:15.812984 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:15.812887    2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:30:15.812984 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.812886    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb3167ad-e92f-4c98-8e23-f68f3f46e399-tmp-dir\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:15.812984 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:15.812897    2545 projected.go:196] Error preparing data for projected volume kube-api-access-sh9h2 for pod openshift-network-diagnostics/network-check-target-fhqvw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:30:15.812984 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.812912    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7716e16f-2459-4ab2-a157-44b83117f9c4-cert\") pod \"ingress-canary-dpm7g\" (UID: \"7716e16f-2459-4ab2-a157-44b83117f9c4\") " pod="openshift-ingress-canary/ingress-canary-dpm7g"
Sep 11 16:30:15.812984 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.812929    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h56vh\" (UniqueName: \"kubernetes.io/projected/7716e16f-2459-4ab2-a157-44b83117f9c4-kube-api-access-h56vh\") pod \"ingress-canary-dpm7g\" (UID: \"7716e16f-2459-4ab2-a157-44b83117f9c4\") " pod="openshift-ingress-canary/ingress-canary-dpm7g"
Sep 11 16:30:15.812984 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.812949    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:15.813238 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:15.812994    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2 podName:eaffdb73-faca-4808-8b86-0783dffb6197 nodeName:}" failed. No retries permitted until 2026-09-11 16:30:47.812971769 +0000 UTC m=+66.366141676 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sh9h2" (UniqueName: "kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2") pod "network-check-target-fhqvw" (UID: "eaffdb73-faca-4808-8b86-0783dffb6197") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:30:15.913435 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.913364    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb3167ad-e92f-4c98-8e23-f68f3f46e399-tmp-dir\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:15.913435 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.913401    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7716e16f-2459-4ab2-a157-44b83117f9c4-cert\") pod \"ingress-canary-dpm7g\" (UID: \"7716e16f-2459-4ab2-a157-44b83117f9c4\") " pod="openshift-ingress-canary/ingress-canary-dpm7g"
Sep 11 16:30:15.913435 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.913427    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h56vh\" (UniqueName: \"kubernetes.io/projected/7716e16f-2459-4ab2-a157-44b83117f9c4-kube-api-access-h56vh\") pod \"ingress-canary-dpm7g\" (UID: \"7716e16f-2459-4ab2-a157-44b83117f9c4\") " pod="openshift-ingress-canary/ingress-canary-dpm7g"
Sep 11 16:30:15.913664 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.913554    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:15.913664 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.913606    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb3167ad-e92f-4c98-8e23-f68f3f46e399-config-volume\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:15.913664 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.913637    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjl74\" (UniqueName: \"kubernetes.io/projected/cb3167ad-e92f-4c98-8e23-f68f3f46e399-kube-api-access-kjl74\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:15.913893 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:15.913691    2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:30:15.913893 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.913726    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb3167ad-e92f-4c98-8e23-f68f3f46e399-tmp-dir\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:15.913893 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:15.913763    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls podName:cb3167ad-e92f-4c98-8e23-f68f3f46e399 nodeName:}" failed. No retries permitted until 2026-09-11 16:30:16.41374417 +0000 UTC m=+34.966914090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls") pod "dns-default-m4w4b" (UID: "cb3167ad-e92f-4c98-8e23-f68f3f46e399") : secret "dns-default-metrics-tls" not found
Sep 11 16:30:15.914258 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.914236    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb3167ad-e92f-4c98-8e23-f68f3f46e399-config-volume\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:15.917991 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.917972    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7716e16f-2459-4ab2-a157-44b83117f9c4-cert\") pod \"ingress-canary-dpm7g\" (UID: \"7716e16f-2459-4ab2-a157-44b83117f9c4\") " pod="openshift-ingress-canary/ingress-canary-dpm7g"
Sep 11 16:30:15.925287 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.925268    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h56vh\" (UniqueName: \"kubernetes.io/projected/7716e16f-2459-4ab2-a157-44b83117f9c4-kube-api-access-h56vh\") pod \"ingress-canary-dpm7g\" (UID: \"7716e16f-2459-4ab2-a157-44b83117f9c4\") " pod="openshift-ingress-canary/ingress-canary-dpm7g"
Sep 11 16:30:15.925991 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.925973    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjl74\" (UniqueName: \"kubernetes.io/projected/cb3167ad-e92f-4c98-8e23-f68f3f46e399-kube-api-access-kjl74\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:15.931710 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.931169    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:30:15.931710 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.931171    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:30:15.931710 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.931475    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:30:15.933559 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.933541    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 16:30:15.933652 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.933611    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 16:30:15.933652 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.933542    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 16:30:15.933759 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.933659    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cczvg\""
Sep 11 16:30:15.934100 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.933954    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 16:30:15.934100 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:15.933957    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5vm65\""
Sep 11 16:30:16.051046 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:16.051018    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dpm7g"
Sep 11 16:30:16.418191 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:16.418158    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:16.418361 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:16.418326    2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:30:16.418438 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:16.418419    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls podName:cb3167ad-e92f-4c98-8e23-f68f3f46e399 nodeName:}" failed. No retries permitted until 2026-09-11 16:30:17.418397139 +0000 UTC m=+35.971567044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls") pod "dns-default-m4w4b" (UID: "cb3167ad-e92f-4c98-8e23-f68f3f46e399") : secret "dns-default-metrics-tls" not found
Sep 11 16:30:17.296143 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:17.296115    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dpm7g"]
Sep 11 16:30:17.299924 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:30:17.299898    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7716e16f_2459_4ab2_a157_44b83117f9c4.slice/crio-670b258a0f90ddb1647efa23ea7443806931f3767bcc257681eb4fb1048202d0 WatchSource:0}: Error finding container 670b258a0f90ddb1647efa23ea7443806931f3767bcc257681eb4fb1048202d0: Status 404 returned error can't find the container with id 670b258a0f90ddb1647efa23ea7443806931f3767bcc257681eb4fb1048202d0
Sep 11 16:30:17.425976 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:17.425838    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:17.426067 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:17.425979    2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:30:17.426067 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:17.426033    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls podName:cb3167ad-e92f-4c98-8e23-f68f3f46e399 nodeName:}" failed. No retries permitted until 2026-09-11 16:30:19.42601945 +0000 UTC m=+37.979189373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls") pod "dns-default-m4w4b" (UID: "cb3167ad-e92f-4c98-8e23-f68f3f46e399") : secret "dns-default-metrics-tls" not found
Sep 11 16:30:18.100143 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:18.100110    2545 generic.go:356] "Generic (PLEG): container finished" podID="189a1a33-c42e-49b2-a086-c7317050492a" containerID="0a5aad7884e9b75ecd22275c2f8ce4052b54d56beac647cbbf3f367c8ce329cb" exitCode=0
Sep 11 16:30:18.100299 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:18.100202    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbtf2" event={"ID":"189a1a33-c42e-49b2-a086-c7317050492a","Type":"ContainerDied","Data":"0a5aad7884e9b75ecd22275c2f8ce4052b54d56beac647cbbf3f367c8ce329cb"}
Sep 11 16:30:18.101480 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:18.101453    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dpm7g" event={"ID":"7716e16f-2459-4ab2-a157-44b83117f9c4","Type":"ContainerStarted","Data":"670b258a0f90ddb1647efa23ea7443806931f3767bcc257681eb4fb1048202d0"}
Sep 11 16:30:19.110203 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:19.110165    2545 generic.go:356] "Generic (PLEG): container finished" podID="189a1a33-c42e-49b2-a086-c7317050492a" containerID="48a50ba0b64aa3ad196fbe044093ca9e213ad1eb82f8d26158d18ea8fdd7511b" exitCode=0
Sep 11 16:30:19.110590 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:19.110198    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbtf2" event={"ID":"189a1a33-c42e-49b2-a086-c7317050492a","Type":"ContainerDied","Data":"48a50ba0b64aa3ad196fbe044093ca9e213ad1eb82f8d26158d18ea8fdd7511b"}
Sep 11 16:30:19.444976 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:19.444892    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:19.445141 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:19.445049    2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:30:19.445141 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:19.445114    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls podName:cb3167ad-e92f-4c98-8e23-f68f3f46e399 nodeName:}" failed. No retries permitted until 2026-09-11 16:30:23.44509923 +0000 UTC m=+41.998269134 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls") pod "dns-default-m4w4b" (UID: "cb3167ad-e92f-4c98-8e23-f68f3f46e399") : secret "dns-default-metrics-tls" not found
Sep 11 16:30:20.115498 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:20.115428    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbtf2" event={"ID":"189a1a33-c42e-49b2-a086-c7317050492a","Type":"ContainerStarted","Data":"56313b21f69564f602ede382946061d533822d56b2fc33729371c15049385e30"}
Sep 11 16:30:20.138622 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:20.138576    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mbtf2" podStartSLOduration=8.19765394 podStartE2EDuration="38.138561631s" totalImagesPullingTime="29.940907691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:29:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:30:20.138353454 +0000 UTC m=+38.691523381" watchObservedRunningTime="2026-09-11 16:30:20.138561631 +0000 UTC m=+38.691731552"
Sep 11 16:30:21.119745 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:21.119708    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dpm7g" event={"ID":"7716e16f-2459-4ab2-a157-44b83117f9c4","Type":"ContainerStarted","Data":"04c2a1fd089525511f1c217fbac00762efdbd35f264ef2e9b757a1ccb657125a"}
Sep 11 16:30:21.142099 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:21.142052    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dpm7g" podStartSLOduration=3.122563315 podStartE2EDuration="6.142032373s" totalImagesPullingTime="3.019469058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:30:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:30:21.141805567 +0000 UTC m=+39.694975505" watchObservedRunningTime="2026-09-11 16:30:21.142032373 +0000 UTC m=+39.695202299"
Sep 11 16:30:23.473033 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:23.472999    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:23.473394 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:23.473138    2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:30:23.473394 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:23.473213    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls podName:cb3167ad-e92f-4c98-8e23-f68f3f46e399 nodeName:}" failed. No retries permitted until 2026-09-11 16:30:31.473197981 +0000 UTC m=+50.026367890 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls") pod "dns-default-m4w4b" (UID: "cb3167ad-e92f-4c98-8e23-f68f3f46e399") : secret "dns-default-metrics-tls" not found
Sep 11 16:30:31.522992 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:31.522946    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:31.523422 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:31.523093    2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:30:31.523422 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:31.523168    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls podName:cb3167ad-e92f-4c98-8e23-f68f3f46e399 nodeName:}" failed. No retries permitted until 2026-09-11 16:30:47.523151342 +0000 UTC m=+66.076321247 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls") pod "dns-default-m4w4b" (UID: "cb3167ad-e92f-4c98-8e23-f68f3f46e399") : secret "dns-default-metrics-tls" not found
Sep 11 16:30:42.095125 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:42.095096    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9jtmq"
Sep 11 16:30:47.529489 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:47.529449    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:30:47.529918 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:47.529571    2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:30:47.529918 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:47.529639    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls podName:cb3167ad-e92f-4c98-8e23-f68f3f46e399 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:19.529626038 +0000 UTC m=+98.082795943 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls") pod "dns-default-m4w4b" (UID: "cb3167ad-e92f-4c98-8e23-f68f3f46e399") : secret "dns-default-metrics-tls" not found
Sep 11 16:30:47.630493 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:47.630452    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs\") pod \"network-metrics-daemon-56kmc\" (UID: \"17b64265-fb83-45fe-820a-c5c6e0dffad7\") " pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:30:47.630622 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:47.630516    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:30:47.632634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:47.632614    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 16:30:47.632784 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:47.632765    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 16:30:47.641381 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:47.641361    2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 16:30:47.641470 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:30:47.641459    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs podName:17b64265-fb83-45fe-820a-c5c6e0dffad7 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:51.641442033 +0000 UTC m=+130.194612135 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs") pod "network-metrics-daemon-56kmc" (UID: "17b64265-fb83-45fe-820a-c5c6e0dffad7") : secret "metrics-daemon-secret" not found
Sep 11 16:30:47.645324 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:47.645291    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf6faf2b-abcc-4abe-bf4f-d7496123108c-original-pull-secret\") pod \"global-pull-secret-syncer-wfmld\" (UID: \"cf6faf2b-abcc-4abe-bf4f-d7496123108c\") " pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:30:47.751245 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:47.751211    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wfmld"
Sep 11 16:30:47.832318 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:47.832288    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh9h2\" (UniqueName: \"kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2\") pod \"network-check-target-fhqvw\" (UID: \"eaffdb73-faca-4808-8b86-0783dffb6197\") " pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:30:47.834677 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:47.834660    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 16:30:47.844664 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:47.844648    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 16:30:47.856500 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:47.856481    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh9h2\" (UniqueName: \"kubernetes.io/projected/eaffdb73-faca-4808-8b86-0783dffb6197-kube-api-access-sh9h2\") pod \"network-check-target-fhqvw\" (UID: \"eaffdb73-faca-4808-8b86-0783dffb6197\") " pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:30:48.027334 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:48.027305    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wfmld"]
Sep 11 16:30:48.031017 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:30:48.030989    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf6faf2b_abcc_4abe_bf4f_d7496123108c.slice/crio-7b602e9cb1f0830a2247b4b1a2b78916fb10c309cab3d8cbbe65c8886eeb1d78 WatchSource:0}: Error finding container 7b602e9cb1f0830a2247b4b1a2b78916fb10c309cab3d8cbbe65c8886eeb1d78: Status 404 returned error can't find the container with id 7b602e9cb1f0830a2247b4b1a2b78916fb10c309cab3d8cbbe65c8886eeb1d78
Sep 11 16:30:48.046323 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:48.046304    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cczvg\""
Sep 11 16:30:48.055162 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:48.055122    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:30:48.185805 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:48.185773    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wfmld" event={"ID":"cf6faf2b-abcc-4abe-bf4f-d7496123108c","Type":"ContainerStarted","Data":"7b602e9cb1f0830a2247b4b1a2b78916fb10c309cab3d8cbbe65c8886eeb1d78"}
Sep 11 16:30:48.329404 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:48.329347    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fhqvw"]
Sep 11 16:30:48.332458 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:30:48.332436    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaffdb73_faca_4808_8b86_0783dffb6197.slice/crio-0c6e5c5f1617fab29c760c8d88882a07520bcac66a10aa376fb39ea302fab38b WatchSource:0}: Error finding container 0c6e5c5f1617fab29c760c8d88882a07520bcac66a10aa376fb39ea302fab38b: Status 404 returned error can't find the container with id 0c6e5c5f1617fab29c760c8d88882a07520bcac66a10aa376fb39ea302fab38b
Sep 11 16:30:49.190705 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:49.190665    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fhqvw" event={"ID":"eaffdb73-faca-4808-8b86-0783dffb6197","Type":"ContainerStarted","Data":"0c6e5c5f1617fab29c760c8d88882a07520bcac66a10aa376fb39ea302fab38b"}
Sep 11 16:30:54.207041 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:54.207007    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wfmld" event={"ID":"cf6faf2b-abcc-4abe-bf4f-d7496123108c","Type":"ContainerStarted","Data":"6ee03bdc26349f3c127285af34cf760334490fdf91ead5ab901b998dcc4f0cf8"}
Sep 11 16:30:54.208648 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:54.208622    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fhqvw" event={"ID":"eaffdb73-faca-4808-8b86-0783dffb6197","Type":"ContainerStarted","Data":"40aaa96f898fae3a9bac35d71186453553f4c64714f81a6e87c009f1e58ec5a3"}
Sep 11 16:30:54.208773 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:54.208755    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:30:54.225675 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:30:54.225632    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wfmld" podStartSLOduration=67.109830306 podStartE2EDuration="1m12.22561917s" totalImagesPullingTime="5.115788864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:30:54.225386532 +0000 UTC m=+72.778556461" watchObservedRunningTime="2026-09-11 16:30:54.22561917 +0000 UTC m=+72.778789097"
Sep 11 16:31:15.587983 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.587825    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fhqvw" podStartSLOduration=88.785113137 podStartE2EDuration="1m33.587810298s" totalImagesPullingTime="4.802697161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:30:54.24007187 +0000 UTC m=+72.793241821" watchObservedRunningTime="2026-09-11 16:31:15.587810298 +0000 UTC m=+94.140980224"
Sep 11 16:31:15.588429 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.588385    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"]
Sep 11 16:31:15.593108 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.593084    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"
Sep 11 16:31:15.595312 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.595290    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 11 16:31:15.595438 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.595344    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zwzkf\""
Sep 11 16:31:15.595896 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.595878    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:31:15.595986 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.595900    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 11 16:31:15.601997 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.601976    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"]
Sep 11 16:31:15.706800 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.706770    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwkm5\" (UniqueName: \"kubernetes.io/projected/562885c5-4c47-41e6-8de2-37a5baab0a38-kube-api-access-vwkm5\") pod \"cluster-samples-operator-86499c89c4-pnptw\" (UID: \"562885c5-4c47-41e6-8de2-37a5baab0a38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"
Sep 11 16:31:15.706954 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.706822    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-pnptw\" (UID: \"562885c5-4c47-41e6-8de2-37a5baab0a38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"
Sep 11 16:31:15.712304 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.712284    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v"]
Sep 11 16:31:15.715007 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.714993    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v"
Sep 11 16:31:15.721830 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.721812    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 11 16:31:15.721949 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.721832    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 11 16:31:15.721949 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.721851    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 11 16:31:15.722846 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.722830    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-v96zh\""
Sep 11 16:31:15.725046 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.725032    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:31:15.727609 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.727589    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-llqgw"]
Sep 11 16:31:15.730176 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.730162    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-vbjs6"]
Sep 11 16:31:15.730297 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.730282    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-llqgw"
Sep 11 16:31:15.733045 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.733029    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:15.737772 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.737732    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-5ngcb\""
Sep 11 16:31:15.737916 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.737784    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 11 16:31:15.737916 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.737741    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 11 16:31:15.737916 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.737782    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:31:15.737916 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.737908    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-n7zl7\""
Sep 11 16:31:15.738268 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.738252    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 11 16:31:15.739078 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.739060    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:31:15.744878 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.744823    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 11 16:31:15.745701 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.745680    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v"]
Sep 11 16:31:15.752090 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.752074    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 11 16:31:15.767903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.767884    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-llqgw"]
Sep 11 16:31:15.792174 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.792152    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-vbjs6"]
Sep 11 16:31:15.807241 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.807218    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwkm5\" (UniqueName: \"kubernetes.io/projected/562885c5-4c47-41e6-8de2-37a5baab0a38-kube-api-access-vwkm5\") pod \"cluster-samples-operator-86499c89c4-pnptw\" (UID: \"562885c5-4c47-41e6-8de2-37a5baab0a38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"
Sep 11 16:31:15.807338 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.807261    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-pnptw\" (UID: \"562885c5-4c47-41e6-8de2-37a5baab0a38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"
Sep 11 16:31:15.807397 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:15.807361    2545 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 16:31:15.807450 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:15.807415    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls podName:562885c5-4c47-41e6-8de2-37a5baab0a38 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:16.307403102 +0000 UTC m=+94.860573020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-pnptw" (UID: "562885c5-4c47-41e6-8de2-37a5baab0a38") : secret "samples-operator-tls" not found
Sep 11 16:31:15.840027 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.839976    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwkm5\" (UniqueName: \"kubernetes.io/projected/562885c5-4c47-41e6-8de2-37a5baab0a38-kube-api-access-vwkm5\") pod \"cluster-samples-operator-86499c89c4-pnptw\" (UID: \"562885c5-4c47-41e6-8de2-37a5baab0a38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"
Sep 11 16:31:15.907880 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.907847    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9d4e405-fb69-425f-b6bb-1e629a25d99f-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-lv54v\" (UID: \"c9d4e405-fb69-425f-b6bb-1e629a25d99f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v"
Sep 11 16:31:15.907979 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.907886    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c10438b1-8196-417e-a982-d7c12e9cdb20-serving-cert\") pod \"console-operator-696884fb9c-vbjs6\" (UID: \"c10438b1-8196-417e-a982-d7c12e9cdb20\") " pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:15.907979 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.907906    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9v4l\" (UniqueName: \"kubernetes.io/projected/8cb42ba7-0e54-493a-ba9a-9ab775943b15-kube-api-access-v9v4l\") pod \"volume-data-source-validator-86695b7cc7-llqgw\" (UID: \"8cb42ba7-0e54-493a-ba9a-9ab775943b15\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-llqgw"
Sep 11 16:31:15.907979 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.907957    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c9d4e405-fb69-425f-b6bb-1e629a25d99f-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-lv54v\" (UID: \"c9d4e405-fb69-425f-b6bb-1e629a25d99f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v"
Sep 11 16:31:15.908089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.907982    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c10438b1-8196-417e-a982-d7c12e9cdb20-config\") pod \"console-operator-696884fb9c-vbjs6\" (UID: \"c10438b1-8196-417e-a982-d7c12e9cdb20\") " pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:15.908089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.908009    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c10438b1-8196-417e-a982-d7c12e9cdb20-trusted-ca\") pod \"console-operator-696884fb9c-vbjs6\" (UID: \"c10438b1-8196-417e-a982-d7c12e9cdb20\") " pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:15.908089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.908047    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwtbb\" (UniqueName: \"kubernetes.io/projected/c9d4e405-fb69-425f-b6bb-1e629a25d99f-kube-api-access-gwtbb\") pod \"kube-storage-version-migrator-operator-6c8f84f897-lv54v\" (UID: \"c9d4e405-fb69-425f-b6bb-1e629a25d99f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v"
Sep 11 16:31:15.908089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:15.908066    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lnn9\" (UniqueName: \"kubernetes.io/projected/c10438b1-8196-417e-a982-d7c12e9cdb20-kube-api-access-5lnn9\") pod \"console-operator-696884fb9c-vbjs6\" (UID: \"c10438b1-8196-417e-a982-d7c12e9cdb20\") " pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:16.008433 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.008407    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9d4e405-fb69-425f-b6bb-1e629a25d99f-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-lv54v\" (UID: \"c9d4e405-fb69-425f-b6bb-1e629a25d99f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v"
Sep 11 16:31:16.008536 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.008434    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c10438b1-8196-417e-a982-d7c12e9cdb20-serving-cert\") pod \"console-operator-696884fb9c-vbjs6\" (UID: \"c10438b1-8196-417e-a982-d7c12e9cdb20\") " pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:16.008536 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.008456    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9v4l\" (UniqueName: \"kubernetes.io/projected/8cb42ba7-0e54-493a-ba9a-9ab775943b15-kube-api-access-v9v4l\") pod \"volume-data-source-validator-86695b7cc7-llqgw\" (UID: \"8cb42ba7-0e54-493a-ba9a-9ab775943b15\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-llqgw"
Sep 11 16:31:16.008536 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.008485    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c9d4e405-fb69-425f-b6bb-1e629a25d99f-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-lv54v\" (UID: \"c9d4e405-fb69-425f-b6bb-1e629a25d99f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v"
Sep 11 16:31:16.008536 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.008501    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c10438b1-8196-417e-a982-d7c12e9cdb20-config\") pod \"console-operator-696884fb9c-vbjs6\" (UID: \"c10438b1-8196-417e-a982-d7c12e9cdb20\") " pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:16.008536 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.008523    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c10438b1-8196-417e-a982-d7c12e9cdb20-trusted-ca\") pod \"console-operator-696884fb9c-vbjs6\" (UID: \"c10438b1-8196-417e-a982-d7c12e9cdb20\") " pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:16.008802 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.008565    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwtbb\" (UniqueName: \"kubernetes.io/projected/c9d4e405-fb69-425f-b6bb-1e629a25d99f-kube-api-access-gwtbb\") pod \"kube-storage-version-migrator-operator-6c8f84f897-lv54v\" (UID: \"c9d4e405-fb69-425f-b6bb-1e629a25d99f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v"
Sep 11 16:31:16.008802 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.008595    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lnn9\" (UniqueName: \"kubernetes.io/projected/c10438b1-8196-417e-a982-d7c12e9cdb20-kube-api-access-5lnn9\") pod \"console-operator-696884fb9c-vbjs6\" (UID: \"c10438b1-8196-417e-a982-d7c12e9cdb20\") " pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:16.009247 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.009196    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c9d4e405-fb69-425f-b6bb-1e629a25d99f-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-lv54v\" (UID: \"c9d4e405-fb69-425f-b6bb-1e629a25d99f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v"
Sep 11 16:31:16.009682 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.009660    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c10438b1-8196-417e-a982-d7c12e9cdb20-config\") pod \"console-operator-696884fb9c-vbjs6\" (UID: \"c10438b1-8196-417e-a982-d7c12e9cdb20\") " pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:16.010033 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.010014    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c10438b1-8196-417e-a982-d7c12e9cdb20-trusted-ca\") pod \"console-operator-696884fb9c-vbjs6\" (UID: \"c10438b1-8196-417e-a982-d7c12e9cdb20\") " pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:16.011145 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.011128    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c10438b1-8196-417e-a982-d7c12e9cdb20-serving-cert\") pod \"console-operator-696884fb9c-vbjs6\" (UID: \"c10438b1-8196-417e-a982-d7c12e9cdb20\") " pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:16.011228 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.011153    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9d4e405-fb69-425f-b6bb-1e629a25d99f-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-lv54v\" (UID: \"c9d4e405-fb69-425f-b6bb-1e629a25d99f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v"
Sep 11 16:31:16.018389 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.018364    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwtbb\" (UniqueName: \"kubernetes.io/projected/c9d4e405-fb69-425f-b6bb-1e629a25d99f-kube-api-access-gwtbb\") pod \"kube-storage-version-migrator-operator-6c8f84f897-lv54v\" (UID: \"c9d4e405-fb69-425f-b6bb-1e629a25d99f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v"
Sep 11 16:31:16.022272 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.022253    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9v4l\" (UniqueName: \"kubernetes.io/projected/8cb42ba7-0e54-493a-ba9a-9ab775943b15-kube-api-access-v9v4l\") pod \"volume-data-source-validator-86695b7cc7-llqgw\" (UID: \"8cb42ba7-0e54-493a-ba9a-9ab775943b15\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-llqgw"
Sep 11 16:31:16.022693 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.022672    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lnn9\" (UniqueName: \"kubernetes.io/projected/c10438b1-8196-417e-a982-d7c12e9cdb20-kube-api-access-5lnn9\") pod \"console-operator-696884fb9c-vbjs6\" (UID: \"c10438b1-8196-417e-a982-d7c12e9cdb20\") " pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:16.022772 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.022753    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v"
Sep 11 16:31:16.038937 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.038910    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-llqgw"
Sep 11 16:31:16.044670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.044645    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:16.297805 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.297764    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v"]
Sep 11 16:31:16.301193 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:31:16.301167    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9d4e405_fb69_425f_b6bb_1e629a25d99f.slice/crio-ff9d55778d65c5f273e7dc39840045ad9f74d6463076d9b86b602f691872c187 WatchSource:0}: Error finding container ff9d55778d65c5f273e7dc39840045ad9f74d6463076d9b86b602f691872c187: Status 404 returned error can't find the container with id ff9d55778d65c5f273e7dc39840045ad9f74d6463076d9b86b602f691872c187
Sep 11 16:31:16.311510 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.311486    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-pnptw\" (UID: \"562885c5-4c47-41e6-8de2-37a5baab0a38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"
Sep 11 16:31:16.311657 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:16.311629    2545 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 16:31:16.311709 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:16.311681    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls podName:562885c5-4c47-41e6-8de2-37a5baab0a38 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:17.311666462 +0000 UTC m=+95.864836380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-pnptw" (UID: "562885c5-4c47-41e6-8de2-37a5baab0a38") : secret "samples-operator-tls" not found
Sep 11 16:31:16.318794 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.318761    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-llqgw"]
Sep 11 16:31:16.330470 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:16.330450    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-vbjs6"]
Sep 11 16:31:16.332950 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:31:16.332929    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc10438b1_8196_417e_a982_d7c12e9cdb20.slice/crio-e14dfaaafc3514cc2f86bb73e971abcbb96f7d64efd249ccc793f5dedc90927b WatchSource:0}: Error finding container e14dfaaafc3514cc2f86bb73e971abcbb96f7d64efd249ccc793f5dedc90927b: Status 404 returned error can't find the container with id e14dfaaafc3514cc2f86bb73e971abcbb96f7d64efd249ccc793f5dedc90927b
Sep 11 16:31:17.267457 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:17.267411    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-vbjs6" event={"ID":"c10438b1-8196-417e-a982-d7c12e9cdb20","Type":"ContainerStarted","Data":"e14dfaaafc3514cc2f86bb73e971abcbb96f7d64efd249ccc793f5dedc90927b"}
Sep 11 16:31:17.268877 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:17.268806    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-llqgw" event={"ID":"8cb42ba7-0e54-493a-ba9a-9ab775943b15","Type":"ContainerStarted","Data":"3f38fa8c39d46afc2066380e0e1fb634591acca2ce7b0e97a39f818912c78499"}
Sep 11 16:31:17.270242 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:17.270217    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v" event={"ID":"c9d4e405-fb69-425f-b6bb-1e629a25d99f","Type":"ContainerStarted","Data":"ff9d55778d65c5f273e7dc39840045ad9f74d6463076d9b86b602f691872c187"}
Sep 11 16:31:17.317691 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:17.317665    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-pnptw\" (UID: \"562885c5-4c47-41e6-8de2-37a5baab0a38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"
Sep 11 16:31:17.317810 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:17.317798    2545 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 16:31:17.317888 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:17.317858    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls podName:562885c5-4c47-41e6-8de2-37a5baab0a38 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:19.31784215 +0000 UTC m=+97.871012071 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-pnptw" (UID: "562885c5-4c47-41e6-8de2-37a5baab0a38") : secret "samples-operator-tls" not found
Sep 11 16:31:18.274813 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:18.274777    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-llqgw" event={"ID":"8cb42ba7-0e54-493a-ba9a-9ab775943b15","Type":"ContainerStarted","Data":"c8c7585c3f9753c47919fa1b417e29024dd4b57ba2b6764d4e1a403263556450"}
Sep 11 16:31:18.292443 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:18.292394    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-llqgw" podStartSLOduration=1.8452894359999998 podStartE2EDuration="3.292379497s" totalImagesPullingTime="1.447090061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:31:18.291571984 +0000 UTC m=+96.844741954" watchObservedRunningTime="2026-09-11 16:31:18.292379497 +0000 UTC m=+96.845549423"
Sep 11 16:31:19.280820 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:19.280787    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v" event={"ID":"c9d4e405-fb69-425f-b6bb-1e629a25d99f","Type":"ContainerStarted","Data":"ce63629f3e12f681733f9e87e9b08657f8866d8d995a71f1198461e22f9b5605"}
Sep 11 16:31:19.282567 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:19.282549    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/0.log"
Sep 11 16:31:19.282655 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:19.282585    2545 generic.go:356] "Generic (PLEG): container finished" podID="c10438b1-8196-417e-a982-d7c12e9cdb20" containerID="d848f3d61fa165fb0db3dc50225d3c081f5efbc0a387aef4369a17666f2e8bb3" exitCode=255
Sep 11 16:31:19.282655 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:19.282644    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-vbjs6" event={"ID":"c10438b1-8196-417e-a982-d7c12e9cdb20","Type":"ContainerDied","Data":"d848f3d61fa165fb0db3dc50225d3c081f5efbc0a387aef4369a17666f2e8bb3"}
Sep 11 16:31:19.282817 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:19.282805    2545 scope.go:117] "RemoveContainer" containerID="d848f3d61fa165fb0db3dc50225d3c081f5efbc0a387aef4369a17666f2e8bb3"
Sep 11 16:31:19.299121 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:19.299071    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v" podStartSLOduration=1.686521821 podStartE2EDuration="4.29905134s" totalImagesPullingTime="2.612529519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:31:19.297856969 +0000 UTC m=+97.851026907" watchObservedRunningTime="2026-09-11 16:31:19.29905134 +0000 UTC m=+97.852221268"
Sep 11 16:31:19.336687 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:19.335849    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-pnptw\" (UID: \"562885c5-4c47-41e6-8de2-37a5baab0a38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"
Sep 11 16:31:19.336687 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:19.336602    2545 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 16:31:19.336687 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:19.336658    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls podName:562885c5-4c47-41e6-8de2-37a5baab0a38 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:23.336638588 +0000 UTC m=+101.889808506 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-pnptw" (UID: "562885c5-4c47-41e6-8de2-37a5baab0a38") : secret "samples-operator-tls" not found
Sep 11 16:31:19.537060 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:19.536967    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:31:19.537223 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:19.537071    2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:31:19.537223 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:19.537130    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls podName:cb3167ad-e92f-4c98-8e23-f68f3f46e399 nodeName:}" failed. No retries permitted until 2026-09-11 16:32:23.537113558 +0000 UTC m=+162.090283468 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls") pod "dns-default-m4w4b" (UID: "cb3167ad-e92f-4c98-8e23-f68f3f46e399") : secret "dns-default-metrics-tls" not found
Sep 11 16:31:20.116809 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.116745    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rm228"]
Sep 11 16:31:20.119903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.119888    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.125858 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.125839    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vrmx6\""
Sep 11 16:31:20.127709 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.127688    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 16:31:20.127807 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.127733    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 16:31:20.127807 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.127782    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 16:31:20.131068 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.131054    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 16:31:20.136166 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.136146    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rm228"]
Sep 11 16:31:20.141586 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.141565    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-data-volume\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.141713 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.141621    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb4wq\" (UniqueName: \"kubernetes.io/projected/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-api-access-bb4wq\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.141713 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.141644    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.141811 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.141716    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-crio-socket\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.141811 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.141754    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.242184 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.242151    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-crio-socket\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.242329 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.242199    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.242329 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.242233    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-data-volume\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.242329 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.242260    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb4wq\" (UniqueName: \"kubernetes.io/projected/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-api-access-bb4wq\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.242329 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.242277    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.242329 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.242287    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-crio-socket\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.242500 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:20.242395    2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:31:20.242500 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:20.242401    2545 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:31:20.242500 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:20.242462    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls podName:616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:20.742442143 +0000 UTC m=+99.295612054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rm228" (UID: "616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:31:20.242605 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:20.242502    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm podName:616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:20.742482999 +0000 UTC m=+99.295652916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rm228" (UID: "616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4") : configmap "kube-rbac-proxy" not found
Sep 11 16:31:20.242605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.242550    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-data-volume\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.254262 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.254240    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb4wq\" (UniqueName: \"kubernetes.io/projected/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-api-access-bb4wq\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.287993 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.287972    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:31:20.288396 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.288379    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/0.log"
Sep 11 16:31:20.288470 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.288417    2545 generic.go:356] "Generic (PLEG): container finished" podID="c10438b1-8196-417e-a982-d7c12e9cdb20" containerID="649d42e85cf39090f8d8d8a5c88b355ae7c849c867c54e50773b305cde0a8e9f" exitCode=255
Sep 11 16:31:20.288524 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.288503    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-vbjs6" event={"ID":"c10438b1-8196-417e-a982-d7c12e9cdb20","Type":"ContainerDied","Data":"649d42e85cf39090f8d8d8a5c88b355ae7c849c867c54e50773b305cde0a8e9f"}
Sep 11 16:31:20.288595 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.288583    2545 scope.go:117] "RemoveContainer" containerID="d848f3d61fa165fb0db3dc50225d3c081f5efbc0a387aef4369a17666f2e8bb3"
Sep 11 16:31:20.288768 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.288738    2545 scope.go:117] "RemoveContainer" containerID="649d42e85cf39090f8d8d8a5c88b355ae7c849c867c54e50773b305cde0a8e9f"
Sep 11 16:31:20.288968 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:20.288942    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-vbjs6_openshift-console-operator(c10438b1-8196-417e-a982-d7c12e9cdb20)\"" pod="openshift-console-operator/console-operator-696884fb9c-vbjs6" podUID="c10438b1-8196-417e-a982-d7c12e9cdb20"
Sep 11 16:31:20.745530 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.745501    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.745667 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:20.745567    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:20.745667 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:20.745632    2545 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:31:20.745735 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:20.745678    2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:31:20.745735 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:20.745705    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm podName:616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:21.745687641 +0000 UTC m=+100.298857552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rm228" (UID: "616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4") : configmap "kube-rbac-proxy" not found
Sep 11 16:31:20.745735 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:20.745723    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls podName:616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:21.745711783 +0000 UTC m=+100.298881705 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rm228" (UID: "616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:31:21.078340 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.078264    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-hxkk2"]
Sep 11 16:31:21.082492 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.082468    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hxkk2"
Sep 11 16:31:21.085137 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.085105    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 16:31:21.085619 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.085604    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 16:31:21.085619 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.085616    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6hrrt\""
Sep 11 16:31:21.094350 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.094328    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-hxkk2"]
Sep 11 16:31:21.148653 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.148625    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th48m\" (UniqueName: \"kubernetes.io/projected/28d927ce-d08b-478a-8cfa-88a8bcfa3159-kube-api-access-th48m\") pod \"migrator-8b4d697c5-hxkk2\" (UID: \"28d927ce-d08b-478a-8cfa-88a8bcfa3159\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hxkk2"
Sep 11 16:31:21.249973 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.249937    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th48m\" (UniqueName: \"kubernetes.io/projected/28d927ce-d08b-478a-8cfa-88a8bcfa3159-kube-api-access-th48m\") pod \"migrator-8b4d697c5-hxkk2\" (UID: \"28d927ce-d08b-478a-8cfa-88a8bcfa3159\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hxkk2"
Sep 11 16:31:21.271155 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.271129    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th48m\" (UniqueName: \"kubernetes.io/projected/28d927ce-d08b-478a-8cfa-88a8bcfa3159-kube-api-access-th48m\") pod \"migrator-8b4d697c5-hxkk2\" (UID: \"28d927ce-d08b-478a-8cfa-88a8bcfa3159\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hxkk2"
Sep 11 16:31:21.292984 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.292964    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:31:21.293294 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.293282    2545 scope.go:117] "RemoveContainer" containerID="649d42e85cf39090f8d8d8a5c88b355ae7c849c867c54e50773b305cde0a8e9f"
Sep 11 16:31:21.293439 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:21.293421    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-vbjs6_openshift-console-operator(c10438b1-8196-417e-a982-d7c12e9cdb20)\"" pod="openshift-console-operator/console-operator-696884fb9c-vbjs6" podUID="c10438b1-8196-417e-a982-d7c12e9cdb20"
Sep 11 16:31:21.391911 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.391821    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hxkk2"
Sep 11 16:31:21.626967 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.626933    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-l9mr4"]
Sep 11 16:31:21.630039 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.630022    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-l9mr4"
Sep 11 16:31:21.632069 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.632047    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-rs77p\""
Sep 11 16:31:21.640727 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.640707    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-l9mr4"]
Sep 11 16:31:21.653713 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.653661    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq9p7\" (UniqueName: \"kubernetes.io/projected/6ced850b-807b-45d8-8b6f-b8790b21ed1f-kube-api-access-zq9p7\") pod \"network-check-source-6f8b768455-l9mr4\" (UID: \"6ced850b-807b-45d8-8b6f-b8790b21ed1f\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-l9mr4"
Sep 11 16:31:21.660265 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.660244    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-hxkk2"]
Sep 11 16:31:21.663961 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:31:21.663928    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28d927ce_d08b_478a_8cfa_88a8bcfa3159.slice/crio-b97de0e1b4f6f891b4e36b951dfcdd1c6a1b9474ec245bfddc70df9c8fc3ddfd WatchSource:0}: Error finding container b97de0e1b4f6f891b4e36b951dfcdd1c6a1b9474ec245bfddc70df9c8fc3ddfd: Status 404 returned error can't find the container with id b97de0e1b4f6f891b4e36b951dfcdd1c6a1b9474ec245bfddc70df9c8fc3ddfd
Sep 11 16:31:21.754942 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.754910    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:21.754942 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.754943    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq9p7\" (UniqueName: \"kubernetes.io/projected/6ced850b-807b-45d8-8b6f-b8790b21ed1f-kube-api-access-zq9p7\") pod \"network-check-source-6f8b768455-l9mr4\" (UID: \"6ced850b-807b-45d8-8b6f-b8790b21ed1f\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-l9mr4"
Sep 11 16:31:21.755140 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:21.755043    2545 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:31:21.755140 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.755088    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:21.755140 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:21.755118    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm podName:616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:23.755095451 +0000 UTC m=+102.308265373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rm228" (UID: "616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4") : configmap "kube-rbac-proxy" not found
Sep 11 16:31:21.755285 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:21.755169    2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:31:21.755285 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:21.755213    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls podName:616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:23.755201962 +0000 UTC m=+102.308371867 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rm228" (UID: "616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:31:21.764399 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.764375    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq9p7\" (UniqueName: \"kubernetes.io/projected/6ced850b-807b-45d8-8b6f-b8790b21ed1f-kube-api-access-zq9p7\") pod \"network-check-source-6f8b768455-l9mr4\" (UID: \"6ced850b-807b-45d8-8b6f-b8790b21ed1f\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-l9mr4"
Sep 11 16:31:21.840418 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.840386    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lxlpg_9350ed8f-0176-49c6-b2c7-92ee69b2c79a/dns-node-resolver/0.log"
Sep 11 16:31:21.938452 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:21.938379    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-l9mr4"
Sep 11 16:31:22.210568 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.210184    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-l9mr4"]
Sep 11 16:31:22.214193 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:31:22.214161    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ced850b_807b_45d8_8b6f_b8790b21ed1f.slice/crio-71da8d7b9704a452dd5737b6819a35242a1e276a79909888d35e7b0a339fc568 WatchSource:0}: Error finding container 71da8d7b9704a452dd5737b6819a35242a1e276a79909888d35e7b0a339fc568: Status 404 returned error can't find the container with id 71da8d7b9704a452dd5737b6819a35242a1e276a79909888d35e7b0a339fc568
Sep 11 16:31:22.298104 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.298068    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hxkk2" event={"ID":"28d927ce-d08b-478a-8cfa-88a8bcfa3159","Type":"ContainerStarted","Data":"b97de0e1b4f6f891b4e36b951dfcdd1c6a1b9474ec245bfddc70df9c8fc3ddfd"}
Sep 11 16:31:22.300122 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.300088    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-l9mr4" event={"ID":"6ced850b-807b-45d8-8b6f-b8790b21ed1f","Type":"ContainerStarted","Data":"7e50813e06b59778bd04111b657dcbf540776e2b3c6f6948af7f0be944c151a0"}
Sep 11 16:31:22.300259 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.300138    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-l9mr4" event={"ID":"6ced850b-807b-45d8-8b6f-b8790b21ed1f","Type":"ContainerStarted","Data":"71da8d7b9704a452dd5737b6819a35242a1e276a79909888d35e7b0a339fc568"}
Sep 11 16:31:22.329088 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.329035    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-l9mr4" podStartSLOduration=1.329017313 podStartE2EDuration="1.329017313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:31:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:31:22.328515095 +0000 UTC m=+100.881685048" watchObservedRunningTime="2026-09-11 16:31:22.329017313 +0000 UTC m=+100.882187242"
Sep 11 16:31:22.841249 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.841227    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8vt9w_5415fcf3-0701-4c3a-8582-6a614a28ab4a/node-ca/0.log"
Sep 11 16:31:22.863586 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.863336    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-9662f"]
Sep 11 16:31:22.867516 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.867502    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-9662f"
Sep 11 16:31:22.869896 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.869851    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 11 16:31:22.869998 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.869853    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 11 16:31:22.869998 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.869920    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 11 16:31:22.869998 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.869945    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 11 16:31:22.870561 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.870544    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vz7l5\""
Sep 11 16:31:22.877562 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.877534    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-9662f"]
Sep 11 16:31:22.964178 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.964149    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5c7cf2b8-abcf-4221-9665-b92599680d5c-signing-cabundle\") pod \"service-ca-568869fdc5-9662f\" (UID: \"5c7cf2b8-abcf-4221-9665-b92599680d5c\") " pod="openshift-service-ca/service-ca-568869fdc5-9662f"
Sep 11 16:31:22.964289 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.964205    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5c7cf2b8-abcf-4221-9665-b92599680d5c-signing-key\") pod \"service-ca-568869fdc5-9662f\" (UID: \"5c7cf2b8-abcf-4221-9665-b92599680d5c\") " pod="openshift-service-ca/service-ca-568869fdc5-9662f"
Sep 11 16:31:22.964348 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:22.964285    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh8q2\" (UniqueName: \"kubernetes.io/projected/5c7cf2b8-abcf-4221-9665-b92599680d5c-kube-api-access-sh8q2\") pod \"service-ca-568869fdc5-9662f\" (UID: \"5c7cf2b8-abcf-4221-9665-b92599680d5c\") " pod="openshift-service-ca/service-ca-568869fdc5-9662f"
Sep 11 16:31:23.064826 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.064746    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5c7cf2b8-abcf-4221-9665-b92599680d5c-signing-cabundle\") pod \"service-ca-568869fdc5-9662f\" (UID: \"5c7cf2b8-abcf-4221-9665-b92599680d5c\") " pod="openshift-service-ca/service-ca-568869fdc5-9662f"
Sep 11 16:31:23.064826 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.064796    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5c7cf2b8-abcf-4221-9665-b92599680d5c-signing-key\") pod \"service-ca-568869fdc5-9662f\" (UID: \"5c7cf2b8-abcf-4221-9665-b92599680d5c\") " pod="openshift-service-ca/service-ca-568869fdc5-9662f"
Sep 11 16:31:23.065050 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.064853    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh8q2\" (UniqueName: \"kubernetes.io/projected/5c7cf2b8-abcf-4221-9665-b92599680d5c-kube-api-access-sh8q2\") pod \"service-ca-568869fdc5-9662f\" (UID: \"5c7cf2b8-abcf-4221-9665-b92599680d5c\") " pod="openshift-service-ca/service-ca-568869fdc5-9662f"
Sep 11 16:31:23.065604 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.065511    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5c7cf2b8-abcf-4221-9665-b92599680d5c-signing-cabundle\") pod \"service-ca-568869fdc5-9662f\" (UID: \"5c7cf2b8-abcf-4221-9665-b92599680d5c\") " pod="openshift-service-ca/service-ca-568869fdc5-9662f"
Sep 11 16:31:23.067296 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.067277    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5c7cf2b8-abcf-4221-9665-b92599680d5c-signing-key\") pod \"service-ca-568869fdc5-9662f\" (UID: \"5c7cf2b8-abcf-4221-9665-b92599680d5c\") " pod="openshift-service-ca/service-ca-568869fdc5-9662f"
Sep 11 16:31:23.074521 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.074498    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh8q2\" (UniqueName: \"kubernetes.io/projected/5c7cf2b8-abcf-4221-9665-b92599680d5c-kube-api-access-sh8q2\") pod \"service-ca-568869fdc5-9662f\" (UID: \"5c7cf2b8-abcf-4221-9665-b92599680d5c\") " pod="openshift-service-ca/service-ca-568869fdc5-9662f"
Sep 11 16:31:23.191410 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.191392    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-9662f"
Sep 11 16:31:23.305219 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.305182    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hxkk2" event={"ID":"28d927ce-d08b-478a-8cfa-88a8bcfa3159","Type":"ContainerStarted","Data":"a0a5d06df8534ad5d18b7cbafab465dd755436c971e00bd650435e0fec02e0e3"}
Sep 11 16:31:23.305624 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.305237    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hxkk2" event={"ID":"28d927ce-d08b-478a-8cfa-88a8bcfa3159","Type":"ContainerStarted","Data":"ccc1348614c16b8a1383cf272d60ac572a3738f19064ea3060701d9cb4f91546"}
Sep 11 16:31:23.324822 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.324744    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hxkk2" podStartSLOduration=1.218775145 podStartE2EDuration="2.324728315s" totalImagesPullingTime="1.10595317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:31:23.324146961 +0000 UTC m=+101.877316910" watchObservedRunningTime="2026-09-11 16:31:23.324728315 +0000 UTC m=+101.877898241"
Sep 11 16:31:23.367348 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.367318    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-pnptw\" (UID: \"562885c5-4c47-41e6-8de2-37a5baab0a38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"
Sep 11 16:31:23.367484 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:23.367412    2545 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 16:31:23.367484 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:23.367467    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls podName:562885c5-4c47-41e6-8de2-37a5baab0a38 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:31.367453458 +0000 UTC m=+109.920623363 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-pnptw" (UID: "562885c5-4c47-41e6-8de2-37a5baab0a38") : secret "samples-operator-tls" not found
Sep 11 16:31:23.470933 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.470909    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-9662f"]
Sep 11 16:31:23.474432 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:31:23.474403    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c7cf2b8_abcf_4221_9665_b92599680d5c.slice/crio-f8bf06c198d33debe3d37c92464cdeade3b05bc55a3b1f572fce66ffc6919e50 WatchSource:0}: Error finding container f8bf06c198d33debe3d37c92464cdeade3b05bc55a3b1f572fce66ffc6919e50: Status 404 returned error can't find the container with id f8bf06c198d33debe3d37c92464cdeade3b05bc55a3b1f572fce66ffc6919e50
Sep 11 16:31:23.651338 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.651261    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dpm7g_7716e16f-2459-4ab2-a157-44b83117f9c4/serve-healthcheck-canary/0.log"
Sep 11 16:31:23.769281 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.769251    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:23.769425 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:23.769311    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:23.769425 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:23.769389    2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:31:23.769425 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:23.769392    2545 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:31:23.769583 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:23.769445    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls podName:616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:27.76943015 +0000 UTC m=+106.322600056 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rm228" (UID: "616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:31:23.769583 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:23.769461    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm podName:616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:27.769453292 +0000 UTC m=+106.322623198 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rm228" (UID: "616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4") : configmap "kube-rbac-proxy" not found
Sep 11 16:31:24.310423 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:24.310384    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-9662f" event={"ID":"5c7cf2b8-abcf-4221-9665-b92599680d5c","Type":"ContainerStarted","Data":"f8bf06c198d33debe3d37c92464cdeade3b05bc55a3b1f572fce66ffc6919e50"}
Sep 11 16:31:25.214472 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:25.214435    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fhqvw"
Sep 11 16:31:26.045358 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:26.045328    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:26.045689 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:26.045376    2545 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:26.045689 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:26.045667    2545 scope.go:117] "RemoveContainer" containerID="649d42e85cf39090f8d8d8a5c88b355ae7c849c867c54e50773b305cde0a8e9f"
Sep 11 16:31:26.045816 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:26.045798    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-vbjs6_openshift-console-operator(c10438b1-8196-417e-a982-d7c12e9cdb20)\"" pod="openshift-console-operator/console-operator-696884fb9c-vbjs6" podUID="c10438b1-8196-417e-a982-d7c12e9cdb20"
Sep 11 16:31:26.318631 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:26.318594    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-9662f" event={"ID":"5c7cf2b8-abcf-4221-9665-b92599680d5c","Type":"ContainerStarted","Data":"4a8ecbdb1f78a956935f54d6687b5fee63529999ca795dfc1ad0347bd7d7e0d9"}
Sep 11 16:31:26.335352 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:26.335311    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-9662f" podStartSLOduration=1.995525126 podStartE2EDuration="4.335296672s" totalImagesPullingTime="2.339771546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:31:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:31:26.334309107 +0000 UTC m=+104.887479040" watchObservedRunningTime="2026-09-11 16:31:26.335296672 +0000 UTC m=+104.888466599"
Sep 11 16:31:27.798063 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:27.798024    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:27.798641 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:27.798113    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:27.798641 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:27.798232    2545 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:31:27.798641 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:27.798298    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm podName:616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:35.798280407 +0000 UTC m=+114.351450318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rm228" (UID: "616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4") : configmap "kube-rbac-proxy" not found
Sep 11 16:31:27.798777 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:27.798694    2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:31:27.798777 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:27.798738    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls podName:616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:35.798724354 +0000 UTC m=+114.351894269 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rm228" (UID: "616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:31:31.422179 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:31.422146    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-pnptw\" (UID: \"562885c5-4c47-41e6-8de2-37a5baab0a38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"
Sep 11 16:31:31.424578 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:31.424556    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/562885c5-4c47-41e6-8de2-37a5baab0a38-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-pnptw\" (UID: \"562885c5-4c47-41e6-8de2-37a5baab0a38\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"
Sep 11 16:31:31.502897 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:31.502853    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"
Sep 11 16:31:31.773972 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:31.773943    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw"]
Sep 11 16:31:32.340159 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:32.340116    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw" event={"ID":"562885c5-4c47-41e6-8de2-37a5baab0a38","Type":"ContainerStarted","Data":"e790703099b307b23ed9d8a6200f5fb138b36c8e598a1bc605b0668c18836197"}
Sep 11 16:31:34.348050 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:34.348017    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw" event={"ID":"562885c5-4c47-41e6-8de2-37a5baab0a38","Type":"ContainerStarted","Data":"3d9d01ae30fed254ba8c0a6cd71969e08c70214641c1018df2655187b420cb0b"}
Sep 11 16:31:34.348482 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:34.348074    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw" event={"ID":"562885c5-4c47-41e6-8de2-37a5baab0a38","Type":"ContainerStarted","Data":"b30bd1a099baec04012b89e533eef3a6de325ed012bf944456a6703da6d7e1b1"}
Sep 11 16:31:34.385768 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:34.385722    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-pnptw" podStartSLOduration=17.454263602 podStartE2EDuration="19.385706086s" totalImagesPullingTime="1.931442484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:31:34.38532206 +0000 UTC m=+112.938491986" watchObservedRunningTime="2026-09-11 16:31:34.385706086 +0000 UTC m=+112.938876012"
Sep 11 16:31:35.854506 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:35.854475    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:35.854970 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:35.854532    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:35.854970 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:35.854633    2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:31:35.854970 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:35.854699    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls podName:616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4 nodeName:}" failed. No retries permitted until 2026-09-11 16:31:51.854683022 +0000 UTC m=+130.407852931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rm228" (UID: "616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:31:35.855085 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:35.855037    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:40.931848 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:40.931806    2545 scope.go:117] "RemoveContainer" containerID="649d42e85cf39090f8d8d8a5c88b355ae7c849c867c54e50773b305cde0a8e9f"
Sep 11 16:31:41.375716 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:41.375693    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:31:41.375859 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:41.375743    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-vbjs6" event={"ID":"c10438b1-8196-417e-a982-d7c12e9cdb20","Type":"ContainerStarted","Data":"6402453dc413dcb507eadb129adf4aed09d12373007bd1afc1edb5c6a9428e8a"}
Sep 11 16:31:41.376065 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:41.376048    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:41.400213 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:41.400171    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-vbjs6" podStartSLOduration=23.822443162 podStartE2EDuration="26.400158042s" totalImagesPullingTime="2.57771488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:31:41.399032833 +0000 UTC m=+119.952202760" watchObservedRunningTime="2026-09-11 16:31:41.400158042 +0000 UTC m=+119.953327977"
Sep 11 16:31:42.358002 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:42.357970    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-vbjs6"
Sep 11 16:31:45.379955 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.379926    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-58957684c5-2wtmf"]
Sep 11 16:31:45.383058 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.383043    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.385150 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.385126    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 16:31:45.385395 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.385132    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 16:31:45.385395 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.385204    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6rbz9\""
Sep 11 16:31:45.385574 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.385238    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 16:31:45.390315 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.390296    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 16:31:45.399271 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.399251    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58957684c5-2wtmf"]
Sep 11 16:31:45.417545 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.417520    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/71509d28-451f-48e2-881f-b1eb1030411c-image-registry-private-configuration\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.417625 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.417557    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/71509d28-451f-48e2-881f-b1eb1030411c-ca-trust-extracted\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.417625 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.417578    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/71509d28-451f-48e2-881f-b1eb1030411c-installation-pull-secrets\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.417625 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.417595    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/71509d28-451f-48e2-881f-b1eb1030411c-registry-tls\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.417724 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.417678    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/71509d28-451f-48e2-881f-b1eb1030411c-bound-sa-token\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.417724 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.417704    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl5mw\" (UniqueName: \"kubernetes.io/projected/71509d28-451f-48e2-881f-b1eb1030411c-kube-api-access-pl5mw\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.417790 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.417740    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/71509d28-451f-48e2-881f-b1eb1030411c-registry-certificates\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.417823 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.417806    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/71509d28-451f-48e2-881f-b1eb1030411c-trusted-ca\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.518289 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.518261    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/71509d28-451f-48e2-881f-b1eb1030411c-registry-certificates\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.518405 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.518298    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/71509d28-451f-48e2-881f-b1eb1030411c-trusted-ca\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.518405 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.518346    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/71509d28-451f-48e2-881f-b1eb1030411c-image-registry-private-configuration\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.518405 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.518384    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/71509d28-451f-48e2-881f-b1eb1030411c-ca-trust-extracted\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.518560 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.518417    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/71509d28-451f-48e2-881f-b1eb1030411c-installation-pull-secrets\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.518560 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.518447    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/71509d28-451f-48e2-881f-b1eb1030411c-registry-tls\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.518560 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.518500    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/71509d28-451f-48e2-881f-b1eb1030411c-bound-sa-token\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.518560 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.518527    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl5mw\" (UniqueName: \"kubernetes.io/projected/71509d28-451f-48e2-881f-b1eb1030411c-kube-api-access-pl5mw\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.518823 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.518804    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/71509d28-451f-48e2-881f-b1eb1030411c-ca-trust-extracted\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.519388 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.519367    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/71509d28-451f-48e2-881f-b1eb1030411c-registry-certificates\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.519592 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.519565    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/71509d28-451f-48e2-881f-b1eb1030411c-trusted-ca\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.521047 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.521027    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/71509d28-451f-48e2-881f-b1eb1030411c-image-registry-private-configuration\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.521146 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.521106    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/71509d28-451f-48e2-881f-b1eb1030411c-registry-tls\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.521203 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.521179    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/71509d28-451f-48e2-881f-b1eb1030411c-installation-pull-secrets\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.546376 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.546349    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl5mw\" (UniqueName: \"kubernetes.io/projected/71509d28-451f-48e2-881f-b1eb1030411c-kube-api-access-pl5mw\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.549521 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.549495    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/71509d28-451f-48e2-881f-b1eb1030411c-bound-sa-token\") pod \"image-registry-58957684c5-2wtmf\" (UID: \"71509d28-451f-48e2-881f-b1eb1030411c\") " pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.693054 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.692976    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:45.978418 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:45.978345    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58957684c5-2wtmf"]
Sep 11 16:31:45.981043 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:31:45.981019    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71509d28_451f_48e2_881f_b1eb1030411c.slice/crio-c07423e03a4f675a49d418033b4b67e8484b83152805b1673e47c1597973be59 WatchSource:0}: Error finding container c07423e03a4f675a49d418033b4b67e8484b83152805b1673e47c1597973be59: Status 404 returned error can't find the container with id c07423e03a4f675a49d418033b4b67e8484b83152805b1673e47c1597973be59
Sep 11 16:31:46.392990 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:46.392950    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58957684c5-2wtmf" event={"ID":"71509d28-451f-48e2-881f-b1eb1030411c","Type":"ContainerStarted","Data":"3e524d28ed25296de26a0cf9de28bccf4064ce5d063f280add55319e34c1c6a6"}
Sep 11 16:31:46.393355 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:46.393006    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58957684c5-2wtmf" event={"ID":"71509d28-451f-48e2-881f-b1eb1030411c","Type":"ContainerStarted","Data":"c07423e03a4f675a49d418033b4b67e8484b83152805b1673e47c1597973be59"}
Sep 11 16:31:46.393355 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:46.393054    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:31:46.418593 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:46.418545    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-58957684c5-2wtmf" podStartSLOduration=1.418530797 podStartE2EDuration="1.418530797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:31:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:31:46.417565385 +0000 UTC m=+124.970735323" watchObservedRunningTime="2026-09-11 16:31:46.418530797 +0000 UTC m=+124.971700724"
Sep 11 16:31:51.661726 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:51.661688    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs\") pod \"network-metrics-daemon-56kmc\" (UID: \"17b64265-fb83-45fe-820a-c5c6e0dffad7\") " pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:31:51.664210 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:51.664175    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17b64265-fb83-45fe-820a-c5c6e0dffad7-metrics-certs\") pod \"network-metrics-daemon-56kmc\" (UID: \"17b64265-fb83-45fe-820a-c5c6e0dffad7\") " pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:31:51.862828 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:51.862800    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:51.865129 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:51.865108    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rm228\" (UID: \"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4\") " pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:51.931969 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:51.931904    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vrmx6\""
Sep 11 16:31:51.940789 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:51.940771    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rm228"
Sep 11 16:31:51.959303 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:51.959284    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5vm65\""
Sep 11 16:31:51.967982 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:51.967964    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56kmc"
Sep 11 16:31:52.219099 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:52.219034    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rm228"]
Sep 11 16:31:52.222377 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:31:52.222350    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod616ec0d0_c00e_4bd8_8590_c56b1a2ba0a4.slice/crio-d527a7b4b875fbfd4388028cfe690537f8d0480f2143b0b8e9c99bb2d51070bd WatchSource:0}: Error finding container d527a7b4b875fbfd4388028cfe690537f8d0480f2143b0b8e9c99bb2d51070bd: Status 404 returned error can't find the container with id d527a7b4b875fbfd4388028cfe690537f8d0480f2143b0b8e9c99bb2d51070bd
Sep 11 16:31:52.244675 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:52.244653    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-56kmc"]
Sep 11 16:31:52.250778 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:31:52.250753    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17b64265_fb83_45fe_820a_c5c6e0dffad7.slice/crio-ab1bed789c31ccf60714c5cb8e303812ccda7626962728dd9732d8381fc23a28 WatchSource:0}: Error finding container ab1bed789c31ccf60714c5cb8e303812ccda7626962728dd9732d8381fc23a28: Status 404 returned error can't find the container with id ab1bed789c31ccf60714c5cb8e303812ccda7626962728dd9732d8381fc23a28
Sep 11 16:31:52.413381 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:52.413346    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-56kmc" event={"ID":"17b64265-fb83-45fe-820a-c5c6e0dffad7","Type":"ContainerStarted","Data":"ab1bed789c31ccf60714c5cb8e303812ccda7626962728dd9732d8381fc23a28"}
Sep 11 16:31:52.414640 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:52.414619    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rm228" event={"ID":"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4","Type":"ContainerStarted","Data":"83389155d531ad9c34326f5bb01be1941aa08a2f804abf3c8355ef95671cab78"}
Sep 11 16:31:52.414745 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:52.414658    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rm228" event={"ID":"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4","Type":"ContainerStarted","Data":"d527a7b4b875fbfd4388028cfe690537f8d0480f2143b0b8e9c99bb2d51070bd"}
Sep 11 16:31:53.419583 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.419501    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rm228" event={"ID":"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4","Type":"ContainerStarted","Data":"0d087b0d7901cdb1b9b4a53e9fefafc02b05e7a8f8cd83f597fb4d1240d46503"}
Sep 11 16:31:53.421215 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.421191    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-56kmc" event={"ID":"17b64265-fb83-45fe-820a-c5c6e0dffad7","Type":"ContainerStarted","Data":"3a7c23577185b9d8187ee7f01a592da68fc282aaf325f94227e496a247d68591"}
Sep 11 16:31:53.421305 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.421234    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-56kmc" event={"ID":"17b64265-fb83-45fe-820a-c5c6e0dffad7","Type":"ContainerStarted","Data":"b30f74b1e22700b36afdb42540fbb90f3163e05c8699fd5297844b3b10ee9203"}
Sep 11 16:31:53.449585 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.449548    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-56kmc" podStartSLOduration=130.547543298 podStartE2EDuration="2m11.449535587s" totalImagesPullingTime="901.992289ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:31:53.447986742 +0000 UTC m=+132.001156669" watchObservedRunningTime="2026-09-11 16:31:53.449535587 +0000 UTC m=+132.002705514"
Sep 11 16:31:53.656211 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.656182    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"]
Sep 11 16:31:53.659451 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.659429    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"
Sep 11 16:31:53.663015 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.662991    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 11 16:31:53.663636 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.663619    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 11 16:31:53.663735 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.663674    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 16:31:53.663947 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.663930    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-kb7ml\""
Sep 11 16:31:53.664231 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.664216    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 16:31:53.664287 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.664245    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 16:31:53.673983 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.673934    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"]
Sep 11 16:31:53.778174 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.778150    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a4a59db-577c-4520-b770-71c8b0bca905-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-nrbh4\" (UID: \"7a4a59db-577c-4520-b770-71c8b0bca905\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"
Sep 11 16:31:53.778309 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.778209    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a4a59db-577c-4520-b770-71c8b0bca905-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-nrbh4\" (UID: \"7a4a59db-577c-4520-b770-71c8b0bca905\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"
Sep 11 16:31:53.778309 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.778233    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr6lm\" (UniqueName: \"kubernetes.io/projected/7a4a59db-577c-4520-b770-71c8b0bca905-kube-api-access-jr6lm\") pod \"prometheus-operator-68bcd7f987-nrbh4\" (UID: \"7a4a59db-577c-4520-b770-71c8b0bca905\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"
Sep 11 16:31:53.778309 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.778281    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a4a59db-577c-4520-b770-71c8b0bca905-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-nrbh4\" (UID: \"7a4a59db-577c-4520-b770-71c8b0bca905\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"
Sep 11 16:31:53.878824 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.878793    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr6lm\" (UniqueName: \"kubernetes.io/projected/7a4a59db-577c-4520-b770-71c8b0bca905-kube-api-access-jr6lm\") pod \"prometheus-operator-68bcd7f987-nrbh4\" (UID: \"7a4a59db-577c-4520-b770-71c8b0bca905\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"
Sep 11 16:31:53.879053 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.878848    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a4a59db-577c-4520-b770-71c8b0bca905-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-nrbh4\" (UID: \"7a4a59db-577c-4520-b770-71c8b0bca905\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"
Sep 11 16:31:53.879053 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.878895    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a4a59db-577c-4520-b770-71c8b0bca905-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-nrbh4\" (UID: \"7a4a59db-577c-4520-b770-71c8b0bca905\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"
Sep 11 16:31:53.879053 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.878970    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a4a59db-577c-4520-b770-71c8b0bca905-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-nrbh4\" (UID: \"7a4a59db-577c-4520-b770-71c8b0bca905\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"
Sep 11 16:31:53.879943 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.879916    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a4a59db-577c-4520-b770-71c8b0bca905-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-nrbh4\" (UID: \"7a4a59db-577c-4520-b770-71c8b0bca905\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"
Sep 11 16:31:53.881919 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.881895    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a4a59db-577c-4520-b770-71c8b0bca905-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-nrbh4\" (UID: \"7a4a59db-577c-4520-b770-71c8b0bca905\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"
Sep 11 16:31:53.882091 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.882070    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a4a59db-577c-4520-b770-71c8b0bca905-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-nrbh4\" (UID: \"7a4a59db-577c-4520-b770-71c8b0bca905\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"
Sep 11 16:31:53.888555 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.888534    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr6lm\" (UniqueName: \"kubernetes.io/projected/7a4a59db-577c-4520-b770-71c8b0bca905-kube-api-access-jr6lm\") pod \"prometheus-operator-68bcd7f987-nrbh4\" (UID: \"7a4a59db-577c-4520-b770-71c8b0bca905\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"
Sep 11 16:31:53.969161 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:53.969101    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"
Sep 11 16:31:54.267165 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:54.267134    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4"]
Sep 11 16:31:54.270300 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:31:54.270275    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a4a59db_577c_4520_b770_71c8b0bca905.slice/crio-2cba3394c04546011e7c8abf8dda87d56f3a551f73ef7e07138e0b77af9085a4 WatchSource:0}: Error finding container 2cba3394c04546011e7c8abf8dda87d56f3a551f73ef7e07138e0b77af9085a4: Status 404 returned error can't find the container with id 2cba3394c04546011e7c8abf8dda87d56f3a551f73ef7e07138e0b77af9085a4
Sep 11 16:31:54.426949 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:54.426914    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4" event={"ID":"7a4a59db-577c-4520-b770-71c8b0bca905","Type":"ContainerStarted","Data":"2cba3394c04546011e7c8abf8dda87d56f3a551f73ef7e07138e0b77af9085a4"}
Sep 11 16:31:55.437744 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:55.437705    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rm228" event={"ID":"616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4","Type":"ContainerStarted","Data":"93420d2d1fba8b2149711b332c7f0b26355f71f24de33015114b70be606114bb"}
Sep 11 16:31:55.477763 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:55.477712    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rm228" podStartSLOduration=33.033761431 podStartE2EDuration="35.47769574s" totalImagesPullingTime="2.443934309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:31:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:31:55.477025087 +0000 UTC m=+134.030195066" watchObservedRunningTime="2026-09-11 16:31:55.47769574 +0000 UTC m=+134.030865667"
Sep 11 16:31:56.444112 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:56.444076    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4" event={"ID":"7a4a59db-577c-4520-b770-71c8b0bca905","Type":"ContainerStarted","Data":"02ad569c7daff8871d21a41e771886ef5e1afab5418c14f8e1a0056d3605df5d"}
Sep 11 16:31:56.444505 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:56.444141    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4" event={"ID":"7a4a59db-577c-4520-b770-71c8b0bca905","Type":"ContainerStarted","Data":"f8931f057dde2068b0edb4efa4bb3619e7e955ad1de5cf7144594e6727f35f4d"}
Sep 11 16:31:56.467725 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:56.467684    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nrbh4" podStartSLOduration=2.082867828 podStartE2EDuration="3.467667836s" totalImagesPullingTime="1.384800008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:31:56.466810826 +0000 UTC m=+135.019980752" watchObservedRunningTime="2026-09-11 16:31:56.467667836 +0000 UTC m=+135.020837763"
Sep 11 16:31:58.468441 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.468411    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c6vf4"]
Sep 11 16:31:58.473127 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.473107    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.476341 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.476322    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-775mx\""
Sep 11 16:31:58.476444 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.476322    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 16:31:58.476911 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.476893    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 16:31:58.477012 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.476939    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 16:31:58.515048 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.515023    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d771d5d4-dde8-4dd0-90ee-999f015e2a96-root\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.515158 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.515065    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.515158 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.515099    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-wtmp\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.515158 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.515126    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-accelerators-collector-config\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.515303 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.515160    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d771d5d4-dde8-4dd0-90ee-999f015e2a96-sys\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.515303 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.515190    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-textfile\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.515303 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.515223    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d771d5d4-dde8-4dd0-90ee-999f015e2a96-metrics-client-ca\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.515303 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.515297    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-tls\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.515436 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.515320    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxxd6\" (UniqueName: \"kubernetes.io/projected/d771d5d4-dde8-4dd0-90ee-999f015e2a96-kube-api-access-dxxd6\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.616052 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.616028    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d771d5d4-dde8-4dd0-90ee-999f015e2a96-root\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.616189 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.616064    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.616189 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.616088    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-wtmp\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.616189 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.616104    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-accelerators-collector-config\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.616189 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.616132    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d771d5d4-dde8-4dd0-90ee-999f015e2a96-sys\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.616189 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.616165    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d771d5d4-dde8-4dd0-90ee-999f015e2a96-root\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.616189 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.616187    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d771d5d4-dde8-4dd0-90ee-999f015e2a96-sys\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.616466 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.616266    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-wtmp\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.616466 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.616290    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-textfile\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.616466 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.616318    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d771d5d4-dde8-4dd0-90ee-999f015e2a96-metrics-client-ca\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.616466 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.616342    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-tls\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.616640 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.616496    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxxd6\" (UniqueName: \"kubernetes.io/projected/d771d5d4-dde8-4dd0-90ee-999f015e2a96-kube-api-access-dxxd6\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.616817 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.616789    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-accelerators-collector-config\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.616979 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.616952    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-textfile\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.617275 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.617114    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d771d5d4-dde8-4dd0-90ee-999f015e2a96-metrics-client-ca\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.618773 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.618750    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.618943 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.618925    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d771d5d4-dde8-4dd0-90ee-999f015e2a96-node-exporter-tls\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.633364 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.633341    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxxd6\" (UniqueName: \"kubernetes.io/projected/d771d5d4-dde8-4dd0-90ee-999f015e2a96-kube-api-access-dxxd6\") pod \"node-exporter-c6vf4\" (UID: \"d771d5d4-dde8-4dd0-90ee-999f015e2a96\") " pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.781504 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:58.781478    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c6vf4"
Sep 11 16:31:58.792959 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:31:58.792932    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd771d5d4_dde8_4dd0_90ee_999f015e2a96.slice/crio-2e0cc44760c8287f2864da0e32bac72be2797d3ed61901a9cc15b2372c9a86ba WatchSource:0}: Error finding container 2e0cc44760c8287f2864da0e32bac72be2797d3ed61901a9cc15b2372c9a86ba: Status 404 returned error can't find the container with id 2e0cc44760c8287f2864da0e32bac72be2797d3ed61901a9cc15b2372c9a86ba
Sep 11 16:31:59.456507 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.456457    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c6vf4" event={"ID":"d771d5d4-dde8-4dd0-90ee-999f015e2a96","Type":"ContainerStarted","Data":"2e0cc44760c8287f2864da0e32bac72be2797d3ed61901a9cc15b2372c9a86ba"}
Sep 11 16:31:59.623650 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.623430    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 16:31:59.627612 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.627585    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.637281 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.637102    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 16:31:59.637281 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.637106    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5c4wr\""
Sep 11 16:31:59.641062 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.640763    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 16:31:59.641062 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.640824    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 16:31:59.641062 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.640823    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 16:31:59.641062 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.640825    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 16:31:59.641062 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.640916    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 16:31:59.641363 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.641285    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 16:31:59.641363 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.641309    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 16:31:59.689021 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.688997    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 16:31:59.724605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.723921    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.724605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.723967    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.724605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.723999    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.724605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.724037    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-web-config\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.724605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.724067    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfp7j\" (UniqueName: \"kubernetes.io/projected/55643449-6537-415d-b51a-86794285a80a-kube-api-access-jfp7j\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.724605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.724099    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55643449-6537-415d-b51a-86794285a80a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.724605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.724149    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-config-volume\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.724605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.724169    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.724605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.724194    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55643449-6537-415d-b51a-86794285a80a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.724605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.724278    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55643449-6537-415d-b51a-86794285a80a-config-out\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.724605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.724324    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55643449-6537-415d-b51a-86794285a80a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.724605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.724348    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.724605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.724375    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55643449-6537-415d-b51a-86794285a80a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.739123 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.739097    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 16:31:59.824943 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.824917    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-config-volume\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.825074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.824948    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.825074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.824970    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55643449-6537-415d-b51a-86794285a80a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.825074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.825033    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55643449-6537-415d-b51a-86794285a80a-config-out\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.825074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.825067    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55643449-6537-415d-b51a-86794285a80a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.825268 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.825089    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.825268 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.825113    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55643449-6537-415d-b51a-86794285a80a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.825268 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.825145    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.825268 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.825168    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.825268 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.825192    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.825268 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:59.825216    2545 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 11 16:31:59.825620 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:31:59.825289    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-main-tls podName:55643449-6537-415d-b51a-86794285a80a nodeName:}" failed. No retries permitted until 2026-09-11 16:32:00.325268429 +0000 UTC m=+138.878438354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "55643449-6537-415d-b51a-86794285a80a") : secret "alertmanager-main-tls" not found
Sep 11 16:31:59.825620 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.825223    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-web-config\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.825620 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.825547    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfp7j\" (UniqueName: \"kubernetes.io/projected/55643449-6537-415d-b51a-86794285a80a-kube-api-access-jfp7j\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.825620 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.825587    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55643449-6537-415d-b51a-86794285a80a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.826051 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.825935    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55643449-6537-415d-b51a-86794285a80a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.826051 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.826095    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55643449-6537-415d-b51a-86794285a80a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.826557 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.826529    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55643449-6537-415d-b51a-86794285a80a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.828664 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.828607    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55643449-6537-415d-b51a-86794285a80a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.828890 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.828842    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-config-volume\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.829037 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.828889    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55643449-6537-415d-b51a-86794285a80a-config-out\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.829108 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.829045    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.829475 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.829447    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.829587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.829499    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.829654 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.829589    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-web-config\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.830266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.830247    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:31:59.856319 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:31:59.856291    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfp7j\" (UniqueName: \"kubernetes.io/projected/55643449-6537-415d-b51a-86794285a80a-kube-api-access-jfp7j\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:32:00.329180 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:00.329098    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:32:00.331672 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:00.331649    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:32:00.461133 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:00.461105    2545 generic.go:356] "Generic (PLEG): container finished" podID="d771d5d4-dde8-4dd0-90ee-999f015e2a96" containerID="f689d64eacc66071163ca69134e78c12368f5d6525fbe0a821866fd2c04ff864" exitCode=0
Sep 11 16:32:00.461257 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:00.461137    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c6vf4" event={"ID":"d771d5d4-dde8-4dd0-90ee-999f015e2a96","Type":"ContainerDied","Data":"f689d64eacc66071163ca69134e78c12368f5d6525fbe0a821866fd2c04ff864"}
Sep 11 16:32:00.539819 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:00.539792    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:32:00.879908 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:32:00.879854    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55643449_6537_415d_b51a_86794285a80a.slice/crio-1855cce492c0c71a3c0323450bef1dd137cf89806bec3d3537a376e5f10e37e1 WatchSource:0}: Error finding container 1855cce492c0c71a3c0323450bef1dd137cf89806bec3d3537a376e5f10e37e1: Status 404 returned error can't find the container with id 1855cce492c0c71a3c0323450bef1dd137cf89806bec3d3537a376e5f10e37e1
Sep 11 16:32:00.881189 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:00.881165    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 16:32:01.466632 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:01.466596    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c6vf4" event={"ID":"d771d5d4-dde8-4dd0-90ee-999f015e2a96","Type":"ContainerStarted","Data":"428fffc0f107a87681a99ef6043ddfd30b763335969bee062a7885c4074731f3"}
Sep 11 16:32:01.466798 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:01.466651    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c6vf4" event={"ID":"d771d5d4-dde8-4dd0-90ee-999f015e2a96","Type":"ContainerStarted","Data":"888cd7a4c3e9ecc0e41840c4bc37b9dedf3c7f784e90a01e50983f56b9149da8"}
Sep 11 16:32:01.467898 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:01.467877    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerStarted","Data":"1855cce492c0c71a3c0323450bef1dd137cf89806bec3d3537a376e5f10e37e1"}
Sep 11 16:32:01.511140 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:01.511084    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c6vf4" podStartSLOduration=2.380861107 podStartE2EDuration="3.511064344s" totalImagesPullingTime="1.062078262s" totalInitContainerRuntime="68.124975ms" isStatefulPod=true podCreationTimestamp="2026-09-11 16:31:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:32:01.508688841 +0000 UTC m=+140.061858818" watchObservedRunningTime="2026-09-11 16:32:01.511064344 +0000 UTC m=+140.064234272"
Sep 11 16:32:02.473355 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:02.473279    2545 generic.go:356] "Generic (PLEG): container finished" podID="55643449-6537-415d-b51a-86794285a80a" containerID="514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c" exitCode=0
Sep 11 16:32:02.473684 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:02.473360    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerDied","Data":"514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c"}
Sep 11 16:32:02.980273 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:02.980240    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-k4758"]
Sep 11 16:32:02.983355 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:02.983339    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k4758"
Sep 11 16:32:02.998964 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:02.998939    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-lkmsr\""
Sep 11 16:32:02.998964 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:02.998951    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 11 16:32:03.008510 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.008491    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-k4758"]
Sep 11 16:32:03.050713 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.050683    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvczn\" (UniqueName: \"kubernetes.io/projected/5b7f295c-29f6-4c32-bd02-16ccc91a2ba5-kube-api-access-nvczn\") pod \"monitoring-plugin-5d85778b9d-k4758\" (UID: \"5b7f295c-29f6-4c32-bd02-16ccc91a2ba5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k4758"
Sep 11 16:32:03.050844 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.050772    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5b7f295c-29f6-4c32-bd02-16ccc91a2ba5-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-k4758\" (UID: \"5b7f295c-29f6-4c32-bd02-16ccc91a2ba5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k4758"
Sep 11 16:32:03.151640 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.151602    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvczn\" (UniqueName: \"kubernetes.io/projected/5b7f295c-29f6-4c32-bd02-16ccc91a2ba5-kube-api-access-nvczn\") pod \"monitoring-plugin-5d85778b9d-k4758\" (UID: \"5b7f295c-29f6-4c32-bd02-16ccc91a2ba5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k4758"
Sep 11 16:32:03.151795 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.151652    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5b7f295c-29f6-4c32-bd02-16ccc91a2ba5-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-k4758\" (UID: \"5b7f295c-29f6-4c32-bd02-16ccc91a2ba5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k4758"
Sep 11 16:32:03.154105 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.154086    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5b7f295c-29f6-4c32-bd02-16ccc91a2ba5-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-k4758\" (UID: \"5b7f295c-29f6-4c32-bd02-16ccc91a2ba5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k4758"
Sep 11 16:32:03.166293 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.166270    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvczn\" (UniqueName: \"kubernetes.io/projected/5b7f295c-29f6-4c32-bd02-16ccc91a2ba5-kube-api-access-nvczn\") pod \"monitoring-plugin-5d85778b9d-k4758\" (UID: \"5b7f295c-29f6-4c32-bd02-16ccc91a2ba5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k4758"
Sep 11 16:32:03.292528 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.292494    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k4758"
Sep 11 16:32:03.294450 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.294423    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-697b76879d-wxkfr"]
Sep 11 16:32:03.297448 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.297433    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.299960 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.299937    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 11 16:32:03.303121 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.303102    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 16:32:03.303225 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.303162    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 11 16:32:03.303225 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.303171    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 11 16:32:03.303342 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.303254    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2ri2n52oskrpb\""
Sep 11 16:32:03.303636 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.303620    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-gt7hb\""
Sep 11 16:32:03.324118 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.324097    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-697b76879d-wxkfr"]
Sep 11 16:32:03.353567 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.353538    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/081b0f85-0ef1-455b-8328-9ca132d8907b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.353672 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.353607    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/081b0f85-0ef1-455b-8328-9ca132d8907b-audit-log\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.353672 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.353636    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/081b0f85-0ef1-455b-8328-9ca132d8907b-metrics-server-audit-profiles\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.353672 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.353661    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/081b0f85-0ef1-455b-8328-9ca132d8907b-client-ca-bundle\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.353853 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.353727    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/081b0f85-0ef1-455b-8328-9ca132d8907b-secret-metrics-server-client-certs\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.353853 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.353772    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kfkt\" (UniqueName: \"kubernetes.io/projected/081b0f85-0ef1-455b-8328-9ca132d8907b-kube-api-access-9kfkt\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.353853 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.353819    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/081b0f85-0ef1-455b-8328-9ca132d8907b-secret-metrics-server-tls\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.455354 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.455311    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kfkt\" (UniqueName: \"kubernetes.io/projected/081b0f85-0ef1-455b-8328-9ca132d8907b-kube-api-access-9kfkt\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.455524 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.455367    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/081b0f85-0ef1-455b-8328-9ca132d8907b-secret-metrics-server-tls\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.455524 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.455431    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/081b0f85-0ef1-455b-8328-9ca132d8907b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.455524 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.455470    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/081b0f85-0ef1-455b-8328-9ca132d8907b-audit-log\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.455524 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.455494    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/081b0f85-0ef1-455b-8328-9ca132d8907b-metrics-server-audit-profiles\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.455524 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.455517    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/081b0f85-0ef1-455b-8328-9ca132d8907b-client-ca-bundle\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.455759 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.455566    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/081b0f85-0ef1-455b-8328-9ca132d8907b-secret-metrics-server-client-certs\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.456908 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.456672    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/081b0f85-0ef1-455b-8328-9ca132d8907b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.456908 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.456829    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/081b0f85-0ef1-455b-8328-9ca132d8907b-metrics-server-audit-profiles\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.457191 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.457168    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/081b0f85-0ef1-455b-8328-9ca132d8907b-audit-log\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.458707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.458663    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/081b0f85-0ef1-455b-8328-9ca132d8907b-secret-metrics-server-client-certs\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.458830 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.458738    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/081b0f85-0ef1-455b-8328-9ca132d8907b-client-ca-bundle\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.459647 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.459624    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/081b0f85-0ef1-455b-8328-9ca132d8907b-secret-metrics-server-tls\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.478530 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.478507    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kfkt\" (UniqueName: \"kubernetes.io/projected/081b0f85-0ef1-455b-8328-9ca132d8907b-kube-api-access-9kfkt\") pod \"metrics-server-697b76879d-wxkfr\" (UID: \"081b0f85-0ef1-455b-8328-9ca132d8907b\") " pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.585855 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.585835    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-k4758"]
Sep 11 16:32:03.587965 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:32:03.587940    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7f295c_29f6_4c32_bd02_16ccc91a2ba5.slice/crio-434ab20f3aa08de90373a2ab3f6e1bf748d260caeb88e1d028462184fbe6896c WatchSource:0}: Error finding container 434ab20f3aa08de90373a2ab3f6e1bf748d260caeb88e1d028462184fbe6896c: Status 404 returned error can't find the container with id 434ab20f3aa08de90373a2ab3f6e1bf748d260caeb88e1d028462184fbe6896c
Sep 11 16:32:03.618008 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.617980    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:03.800553 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.800520    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-66649b79bd-4jr7t"]
Sep 11 16:32:03.804264 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.804245    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.808715 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.808589    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 11 16:32:03.808715 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.808651    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 11 16:32:03.808907 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.808720    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 11 16:32:03.809248 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.809114    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 11 16:32:03.809248 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.809124    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 11 16:32:03.817528 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.816100    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-smplr\""
Sep 11 16:32:03.834580 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.834553    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 11 16:32:03.836202 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.836138    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-66649b79bd-4jr7t"]
Sep 11 16:32:03.860141 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.860107    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf6691ec-f9d1-4e88-b861-23e617102881-serving-certs-ca-bundle\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.860259 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.860174    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf6691ec-f9d1-4e88-b861-23e617102881-metrics-client-ca\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.860259 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.860203    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cf6691ec-f9d1-4e88-b861-23e617102881-telemeter-client-tls\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.860339 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.860317    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpcgg\" (UniqueName: \"kubernetes.io/projected/cf6691ec-f9d1-4e88-b861-23e617102881-kube-api-access-wpcgg\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.860383 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.860366    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf6691ec-f9d1-4e88-b861-23e617102881-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.860469 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.860448    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf6691ec-f9d1-4e88-b861-23e617102881-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.860531 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.860503    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cf6691ec-f9d1-4e88-b861-23e617102881-federate-client-tls\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.860588 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.860549    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cf6691ec-f9d1-4e88-b861-23e617102881-secret-telemeter-client\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.884548 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.884522    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-df544d489-cth5d"]
Sep 11 16:32:03.887721 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.887703    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:03.890977 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.890954    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 16:32:03.891067 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.890984    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 16:32:03.891587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.891247    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 16:32:03.891587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.891321    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 16:32:03.891587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.891477    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 16:32:03.891587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.891501    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 16:32:03.891587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.891511    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-p2f27\""
Sep 11 16:32:03.891930 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.891915    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 16:32:03.896409 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.896390    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 16:32:03.910547 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.910526    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-df544d489-cth5d"]
Sep 11 16:32:03.935893 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:32:03.935852    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081b0f85_0ef1_455b_8328_9ca132d8907b.slice/crio-ed33136213bb2e07e21b9073c0c46e4313ee8584f084379a128f7a3dba5f1f1b WatchSource:0}: Error finding container ed33136213bb2e07e21b9073c0c46e4313ee8584f084379a128f7a3dba5f1f1b: Status 404 returned error can't find the container with id ed33136213bb2e07e21b9073c0c46e4313ee8584f084379a128f7a3dba5f1f1b
Sep 11 16:32:03.938050 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.938020    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-697b76879d-wxkfr"]
Sep 11 16:32:03.961203 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961169    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf6691ec-f9d1-4e88-b861-23e617102881-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.961316 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961214    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-serving-cert\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:03.961316 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961245    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-service-ca\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:03.961316 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961285    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf6691ec-f9d1-4e88-b861-23e617102881-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.961316 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961312    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml8kt\" (UniqueName: \"kubernetes.io/projected/7545e6d5-d3e6-4bd8-9455-bdad7e094513-kube-api-access-ml8kt\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:03.961515 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961343    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cf6691ec-f9d1-4e88-b861-23e617102881-federate-client-tls\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.961515 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961374    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-trusted-ca-bundle\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:03.961515 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961429    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cf6691ec-f9d1-4e88-b861-23e617102881-secret-telemeter-client\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.961515 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961471    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf6691ec-f9d1-4e88-b861-23e617102881-serving-certs-ca-bundle\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.961515 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961503    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-oauth-config\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:03.961736 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961531    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-oauth-serving-cert\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:03.961736 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961568    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf6691ec-f9d1-4e88-b861-23e617102881-metrics-client-ca\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.961736 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961606    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cf6691ec-f9d1-4e88-b861-23e617102881-telemeter-client-tls\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.961736 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961683    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpcgg\" (UniqueName: \"kubernetes.io/projected/cf6691ec-f9d1-4e88-b861-23e617102881-kube-api-access-wpcgg\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.961736 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.961713    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-config\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:03.962413 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.962296    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf6691ec-f9d1-4e88-b861-23e617102881-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.962514 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.962487    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf6691ec-f9d1-4e88-b861-23e617102881-metrics-client-ca\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.962659 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.962636    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf6691ec-f9d1-4e88-b861-23e617102881-serving-certs-ca-bundle\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.964500 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.964476    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf6691ec-f9d1-4e88-b861-23e617102881-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.964740 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.964719    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cf6691ec-f9d1-4e88-b861-23e617102881-federate-client-tls\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.964794 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.964769    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cf6691ec-f9d1-4e88-b861-23e617102881-telemeter-client-tls\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.964960 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.964943    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cf6691ec-f9d1-4e88-b861-23e617102881-secret-telemeter-client\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:03.978776 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:03.978754    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpcgg\" (UniqueName: \"kubernetes.io/projected/cf6691ec-f9d1-4e88-b861-23e617102881-kube-api-access-wpcgg\") pod \"telemeter-client-66649b79bd-4jr7t\" (UID: \"cf6691ec-f9d1-4e88-b861-23e617102881\") " pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:04.062481 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.062450    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-config\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.062604 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.062496    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-serving-cert\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.062604 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.062517    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-service-ca\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.062734 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.062597    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml8kt\" (UniqueName: \"kubernetes.io/projected/7545e6d5-d3e6-4bd8-9455-bdad7e094513-kube-api-access-ml8kt\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.062734 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.062650    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-trusted-ca-bundle\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.062734 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.062722    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-oauth-config\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.062894 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.062774    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-oauth-serving-cert\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.063497 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.063469    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-oauth-serving-cert\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.063605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.063500    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-config\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.063605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.063600    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-trusted-ca-bundle\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.064090 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.064043    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-service-ca\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.065373 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.065329    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-serving-cert\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.065515 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.065495    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-oauth-config\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.080230 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.080208    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml8kt\" (UniqueName: \"kubernetes.io/projected/7545e6d5-d3e6-4bd8-9455-bdad7e094513-kube-api-access-ml8kt\") pod \"console-df544d489-cth5d\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") " pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.119663 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.119606    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t"
Sep 11 16:32:04.198202 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.198174    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:04.481954 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.481920    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerStarted","Data":"a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca"}
Sep 11 16:32:04.483474 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.483439    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k4758" event={"ID":"5b7f295c-29f6-4c32-bd02-16ccc91a2ba5","Type":"ContainerStarted","Data":"434ab20f3aa08de90373a2ab3f6e1bf748d260caeb88e1d028462184fbe6896c"}
Sep 11 16:32:04.484625 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.484601    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-697b76879d-wxkfr" event={"ID":"081b0f85-0ef1-455b-8328-9ca132d8907b","Type":"ContainerStarted","Data":"ed33136213bb2e07e21b9073c0c46e4313ee8584f084379a128f7a3dba5f1f1b"}
Sep 11 16:32:04.612059 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:32:04.611997    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7545e6d5_d3e6_4bd8_9455_bdad7e094513.slice/crio-0bb3aa5ca8f125afd684d07620764f5e3183c6e6302231a2e9c14f5d780c93e2 WatchSource:0}: Error finding container 0bb3aa5ca8f125afd684d07620764f5e3183c6e6302231a2e9c14f5d780c93e2: Status 404 returned error can't find the container with id 0bb3aa5ca8f125afd684d07620764f5e3183c6e6302231a2e9c14f5d780c93e2
Sep 11 16:32:04.613893 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.613850    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-df544d489-cth5d"]
Sep 11 16:32:04.619517 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:04.619492    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-66649b79bd-4jr7t"]
Sep 11 16:32:04.623139 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:32:04.623088    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf6691ec_f9d1_4e88_b861_23e617102881.slice/crio-e875a3b77eacae5ae3030e42b37f5fc7a2e99c1b883bb3a68a3884a81b36f146 WatchSource:0}: Error finding container e875a3b77eacae5ae3030e42b37f5fc7a2e99c1b883bb3a68a3884a81b36f146: Status 404 returned error can't find the container with id e875a3b77eacae5ae3030e42b37f5fc7a2e99c1b883bb3a68a3884a81b36f146
Sep 11 16:32:05.495263 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:05.495226    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerStarted","Data":"62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25"}
Sep 11 16:32:05.495727 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:05.495313    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerStarted","Data":"046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848"}
Sep 11 16:32:05.495727 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:05.495355    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerStarted","Data":"d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16"}
Sep 11 16:32:05.495727 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:05.495398    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerStarted","Data":"6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd"}
Sep 11 16:32:05.504818 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:05.504728    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t" event={"ID":"cf6691ec-f9d1-4e88-b861-23e617102881","Type":"ContainerStarted","Data":"e875a3b77eacae5ae3030e42b37f5fc7a2e99c1b883bb3a68a3884a81b36f146"}
Sep 11 16:32:05.506658 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:05.506601    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df544d489-cth5d" event={"ID":"7545e6d5-d3e6-4bd8-9455-bdad7e094513","Type":"ContainerStarted","Data":"0bb3aa5ca8f125afd684d07620764f5e3183c6e6302231a2e9c14f5d780c93e2"}
Sep 11 16:32:07.401629 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:07.401599    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-58957684c5-2wtmf"
Sep 11 16:32:08.526823 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.526790    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerStarted","Data":"62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae"}
Sep 11 16:32:08.529101 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.529069    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t" event={"ID":"cf6691ec-f9d1-4e88-b861-23e617102881","Type":"ContainerStarted","Data":"1bc6e55581be4fe5c02033606f8b8ab81e72eb46eb4b02e4d20ea6da992ff46f"}
Sep 11 16:32:08.529225 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.529131    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t" event={"ID":"cf6691ec-f9d1-4e88-b861-23e617102881","Type":"ContainerStarted","Data":"202eb4146b1e173af77b65a436ae29c0fa2f8619d18ffceffc6221f09da1a301"}
Sep 11 16:32:08.529225 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.529163    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t" event={"ID":"cf6691ec-f9d1-4e88-b861-23e617102881","Type":"ContainerStarted","Data":"2acf072335942893f2ece379b7b4360488509e6d1496d8cc0dc58be73f8fec80"}
Sep 11 16:32:08.530698 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.530671    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df544d489-cth5d" event={"ID":"7545e6d5-d3e6-4bd8-9455-bdad7e094513","Type":"ContainerStarted","Data":"1adc1e7f60ef63dfb6b040c635e929ad1bc6ad691888d34168b6375989ce486a"}
Sep 11 16:32:08.532241 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.532218    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k4758" event={"ID":"5b7f295c-29f6-4c32-bd02-16ccc91a2ba5","Type":"ContainerStarted","Data":"369182bc3a98843f274d9a380d0e1dbd58ff7e2b0cfbc482223cde77ab487aba"}
Sep 11 16:32:08.532388 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.532370    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k4758"
Sep 11 16:32:08.533993 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.533971    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-697b76879d-wxkfr" event={"ID":"081b0f85-0ef1-455b-8328-9ca132d8907b","Type":"ContainerStarted","Data":"98fd8232202c12a91d0752b7230de27966b2e608b87185588a94d757fd6d74b6"}
Sep 11 16:32:08.537344 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.537329    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k4758"
Sep 11 16:32:08.560875 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.560824    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.139921907 podStartE2EDuration="9.560810988s" totalImagesPullingTime="6.399950929s" totalInitContainerRuntime="20.938152ms" isStatefulPod=false podCreationTimestamp="2026-09-11 16:31:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:32:08.559391922 +0000 UTC m=+147.112561849" watchObservedRunningTime="2026-09-11 16:32:08.560810988 +0000 UTC m=+147.113980915"
Sep 11 16:32:08.581341 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.581279    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-697b76879d-wxkfr" podStartSLOduration=1.5182718849999999 podStartE2EDuration="5.581260379s" totalImagesPullingTime="4.062988494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:32:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:32:08.58052375 +0000 UTC m=+147.133693703" watchObservedRunningTime="2026-09-11 16:32:08.581260379 +0000 UTC m=+147.134430307"
Sep 11 16:32:08.606019 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.605968    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-66649b79bd-4jr7t" podStartSLOduration=2.228737962 podStartE2EDuration="5.605950429s" totalImagesPullingTime="3.377212467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:32:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:32:08.603691364 +0000 UTC m=+147.156861290" watchObservedRunningTime="2026-09-11 16:32:08.605950429 +0000 UTC m=+147.159120358"
Sep 11 16:32:08.633448 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.632635    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-k4758" podStartSLOduration=2.221798719 podStartE2EDuration="6.632620278s" totalImagesPullingTime="4.410821559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:32:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:32:08.631533551 +0000 UTC m=+147.184703498" watchObservedRunningTime="2026-09-11 16:32:08.632620278 +0000 UTC m=+147.185790207"
Sep 11 16:32:08.656230 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.656178    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-df544d489-cth5d" podStartSLOduration=2.252384462 podStartE2EDuration="5.656165079s" totalImagesPullingTime="3.403780617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:32:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:32:08.655354345 +0000 UTC m=+147.208524272" watchObservedRunningTime="2026-09-11 16:32:08.656165079 +0000 UTC m=+147.209335006"
Sep 11 16:32:08.745119 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:08.745086    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-df544d489-cth5d"]
Sep 11 16:32:14.198809 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:14.198768    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:18.733567 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:32:18.733521    2545 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-m4w4b" podUID="cb3167ad-e92f-4c98-8e23-f68f3f46e399"
Sep 11 16:32:19.583675 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:19.583642    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:32:23.618467 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:23.618437    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:23.618939 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:23.618491    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:23.629024 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:23.628998    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:32:23.631445 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:23.631425    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3167ad-e92f-4c98-8e23-f68f3f46e399-metrics-tls\") pod \"dns-default-m4w4b\" (UID: \"cb3167ad-e92f-4c98-8e23-f68f3f46e399\") " pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:32:23.786486 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:23.786457    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-55n2r\""
Sep 11 16:32:23.795378 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:23.795361    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:32:24.072777 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:24.072756    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m4w4b"]
Sep 11 16:32:24.075138 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:32:24.075109    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3167ad_e92f_4c98_8e23_f68f3f46e399.slice/crio-fe6bc9397f67edc3f8b74bded30034899e458105edc2f2db5b27186652a9e0f4 WatchSource:0}: Error finding container fe6bc9397f67edc3f8b74bded30034899e458105edc2f2db5b27186652a9e0f4: Status 404 returned error can't find the container with id fe6bc9397f67edc3f8b74bded30034899e458105edc2f2db5b27186652a9e0f4
Sep 11 16:32:24.606514 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:24.606477    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4w4b" event={"ID":"cb3167ad-e92f-4c98-8e23-f68f3f46e399","Type":"ContainerStarted","Data":"fe6bc9397f67edc3f8b74bded30034899e458105edc2f2db5b27186652a9e0f4"}
Sep 11 16:32:26.617654 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:26.617623    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4w4b" event={"ID":"cb3167ad-e92f-4c98-8e23-f68f3f46e399","Type":"ContainerStarted","Data":"37216bac174c18135fd72e1a1226570d8d3d9e960ab7893d5db10f575b71c8dd"}
Sep 11 16:32:26.618088 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:26.617674    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4w4b" event={"ID":"cb3167ad-e92f-4c98-8e23-f68f3f46e399","Type":"ContainerStarted","Data":"5998b99b5bde6f5e5426c4a5bb94fbf4b164d8d4b2d67dcde5edaa0043b0a00b"}
Sep 11 16:32:26.618088 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:26.617746    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:32:26.624209 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:26.624180    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lxlpg_9350ed8f-0176-49c6-b2c7-92ee69b2c79a/dns-node-resolver/0.log"
Sep 11 16:32:26.638379 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:26.638340    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-m4w4b" podStartSLOduration=130.169697706 podStartE2EDuration="2m11.638327687s" totalImagesPullingTime="1.468629981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:30:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:32:26.637618462 +0000 UTC m=+165.190788404" watchObservedRunningTime="2026-09-11 16:32:26.638327687 +0000 UTC m=+165.191497614"
Sep 11 16:32:35.563671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.563624    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-df544d489-cth5d" podUID="7545e6d5-d3e6-4bd8-9455-bdad7e094513" containerName="console" containerID="cri-o://1adc1e7f60ef63dfb6b040c635e929ad1bc6ad691888d34168b6375989ce486a" gracePeriod=15
Sep 11 16:32:35.655593 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.655564    2545 generic.go:356] "Generic (PLEG): container finished" podID="c9d4e405-fb69-425f-b6bb-1e629a25d99f" containerID="ce63629f3e12f681733f9e87e9b08657f8866d8d995a71f1198461e22f9b5605" exitCode=0
Sep 11 16:32:35.655720 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.655604    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v" event={"ID":"c9d4e405-fb69-425f-b6bb-1e629a25d99f","Type":"ContainerDied","Data":"ce63629f3e12f681733f9e87e9b08657f8866d8d995a71f1198461e22f9b5605"}
Sep 11 16:32:35.655977 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.655961    2545 scope.go:117] "RemoveContainer" containerID="ce63629f3e12f681733f9e87e9b08657f8866d8d995a71f1198461e22f9b5605"
Sep 11 16:32:35.814730 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.814675    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:35.921514 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.921477    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-trusted-ca-bundle\") pod \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") "
Sep 11 16:32:35.921656 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.921588    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-oauth-config\") pod \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") "
Sep 11 16:32:35.921656 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.921623    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-service-ca\") pod \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") "
Sep 11 16:32:35.921656 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.921652    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-oauth-serving-cert\") pod \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") "
Sep 11 16:32:35.921844 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.921807    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ml8kt\" (UniqueName: \"kubernetes.io/projected/7545e6d5-d3e6-4bd8-9455-bdad7e094513-kube-api-access-ml8kt\") pod \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") "
Sep 11 16:32:35.921947 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.921858    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-config\") pod \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") "
Sep 11 16:32:35.921947 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.921922    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-serving-cert\") pod \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\" (UID: \"7545e6d5-d3e6-4bd8-9455-bdad7e094513\") "
Sep 11 16:32:35.922052 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.921944    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7545e6d5-d3e6-4bd8-9455-bdad7e094513" (UID: "7545e6d5-d3e6-4bd8-9455-bdad7e094513"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:32:35.922052 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.922010    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-service-ca" (OuterVolumeSpecName: "service-ca") pod "7545e6d5-d3e6-4bd8-9455-bdad7e094513" (UID: "7545e6d5-d3e6-4bd8-9455-bdad7e094513"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:32:35.922052 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.922027    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7545e6d5-d3e6-4bd8-9455-bdad7e094513" (UID: "7545e6d5-d3e6-4bd8-9455-bdad7e094513"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:32:35.922200 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.922182    2545 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:32:35.922257 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.922200    2545 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-oauth-serving-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:32:35.922257 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.922216    2545 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-trusted-ca-bundle\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:32:35.922257 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.922248    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-config" (OuterVolumeSpecName: "console-config") pod "7545e6d5-d3e6-4bd8-9455-bdad7e094513" (UID: "7545e6d5-d3e6-4bd8-9455-bdad7e094513"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:32:35.924146 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.924119    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7545e6d5-d3e6-4bd8-9455-bdad7e094513" (UID: "7545e6d5-d3e6-4bd8-9455-bdad7e094513"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:32:35.924246 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.924188    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7545e6d5-d3e6-4bd8-9455-bdad7e094513-kube-api-access-ml8kt" (OuterVolumeSpecName: "kube-api-access-ml8kt") pod "7545e6d5-d3e6-4bd8-9455-bdad7e094513" (UID: "7545e6d5-d3e6-4bd8-9455-bdad7e094513"). InnerVolumeSpecName "kube-api-access-ml8kt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:32:35.924342 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:35.924327    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7545e6d5-d3e6-4bd8-9455-bdad7e094513" (UID: "7545e6d5-d3e6-4bd8-9455-bdad7e094513"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:32:36.023149 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.023120    2545 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-oauth-config\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:32:36.023149 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.023148    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ml8kt\" (UniqueName: \"kubernetes.io/projected/7545e6d5-d3e6-4bd8-9455-bdad7e094513-kube-api-access-ml8kt\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:32:36.023279 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.023164    2545 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-config\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:32:36.023279 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.023177    2545 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7545e6d5-d3e6-4bd8-9455-bdad7e094513-console-serving-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:32:36.624385 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.624354    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-m4w4b"
Sep 11 16:32:36.661443 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.661410    2545 generic.go:356] "Generic (PLEG): container finished" podID="7545e6d5-d3e6-4bd8-9455-bdad7e094513" containerID="1adc1e7f60ef63dfb6b040c635e929ad1bc6ad691888d34168b6375989ce486a" exitCode=0
Sep 11 16:32:36.661443 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.661444    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df544d489-cth5d"
Sep 11 16:32:36.661658 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.661485    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df544d489-cth5d" event={"ID":"7545e6d5-d3e6-4bd8-9455-bdad7e094513","Type":"ContainerDied","Data":"1adc1e7f60ef63dfb6b040c635e929ad1bc6ad691888d34168b6375989ce486a"}
Sep 11 16:32:36.661658 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.661532    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df544d489-cth5d" event={"ID":"7545e6d5-d3e6-4bd8-9455-bdad7e094513","Type":"ContainerDied","Data":"0bb3aa5ca8f125afd684d07620764f5e3183c6e6302231a2e9c14f5d780c93e2"}
Sep 11 16:32:36.661658 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.661557    2545 scope.go:117] "RemoveContainer" containerID="1adc1e7f60ef63dfb6b040c635e929ad1bc6ad691888d34168b6375989ce486a"
Sep 11 16:32:36.663790 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.663761    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-lv54v" event={"ID":"c9d4e405-fb69-425f-b6bb-1e629a25d99f","Type":"ContainerStarted","Data":"098fbe25bb021ed5e4252992b7f6c5d3ce0f167909778283ff3a03d221732b75"}
Sep 11 16:32:36.671242 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.671222    2545 scope.go:117] "RemoveContainer" containerID="1adc1e7f60ef63dfb6b040c635e929ad1bc6ad691888d34168b6375989ce486a"
Sep 11 16:32:36.671554 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:32:36.671527    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1adc1e7f60ef63dfb6b040c635e929ad1bc6ad691888d34168b6375989ce486a\": container with ID starting with 1adc1e7f60ef63dfb6b040c635e929ad1bc6ad691888d34168b6375989ce486a not found: ID does not exist" containerID="1adc1e7f60ef63dfb6b040c635e929ad1bc6ad691888d34168b6375989ce486a"
Sep 11 16:32:36.671644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.671562    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1adc1e7f60ef63dfb6b040c635e929ad1bc6ad691888d34168b6375989ce486a"} err="failed to get container status \"1adc1e7f60ef63dfb6b040c635e929ad1bc6ad691888d34168b6375989ce486a\": rpc error: code = NotFound desc = could not find container \"1adc1e7f60ef63dfb6b040c635e929ad1bc6ad691888d34168b6375989ce486a\": container with ID starting with 1adc1e7f60ef63dfb6b040c635e929ad1bc6ad691888d34168b6375989ce486a not found: ID does not exist"
Sep 11 16:32:36.744699 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.744670    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-df544d489-cth5d"]
Sep 11 16:32:36.764108 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:36.764081    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-df544d489-cth5d"]
Sep 11 16:32:37.937161 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:37.937126    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7545e6d5-d3e6-4bd8-9455-bdad7e094513" path="/var/lib/kubelet/pods/7545e6d5-d3e6-4bd8-9455-bdad7e094513/volumes"
Sep 11 16:32:43.624491 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:43.624462    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:32:43.628586 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:32:43.628560    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-697b76879d-wxkfr"
Sep 11 16:33:16.392615 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.392576    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-695c5b8ff5-hsvqr"]
Sep 11 16:33:16.393187 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.393051    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7545e6d5-d3e6-4bd8-9455-bdad7e094513" containerName="console"
Sep 11 16:33:16.393187 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.393067    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="7545e6d5-d3e6-4bd8-9455-bdad7e094513" containerName="console"
Sep 11 16:33:16.393187 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.393129    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="7545e6d5-d3e6-4bd8-9455-bdad7e094513" containerName="console"
Sep 11 16:33:16.397239 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.397216    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.399718 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.399695    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 16:33:16.399990 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.399970    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 16:33:16.400079 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.400037    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 16:33:16.400144 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.400103    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 16:33:16.400219 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.400203    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-p2f27\""
Sep 11 16:33:16.400276 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.400216    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 16:33:16.400276 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.400220    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 16:33:16.400385 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.400285    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 16:33:16.409767 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.409728    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-695c5b8ff5-hsvqr"]
Sep 11 16:33:16.411674 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.411648    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 16:33:16.511021 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.510978    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-trusted-ca-bundle\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.511021 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.511024    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14b3a283-8e07-4350-8e92-c0044d151af3-console-oauth-config\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.511222 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.511114    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-service-ca\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.511222 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.511146    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14b3a283-8e07-4350-8e92-c0044d151af3-console-serving-cert\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.511222 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.511182    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-oauth-serving-cert\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.511222 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.511214    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-console-config\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.511388 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.511228    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq47p\" (UniqueName: \"kubernetes.io/projected/14b3a283-8e07-4350-8e92-c0044d151af3-kube-api-access-xq47p\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.612328 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.612291    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14b3a283-8e07-4350-8e92-c0044d151af3-console-oauth-config\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.612498 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.612348    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-service-ca\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.612498 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.612386    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14b3a283-8e07-4350-8e92-c0044d151af3-console-serving-cert\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.612498 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.612424    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-oauth-serving-cert\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.612498 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.612456    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-console-config\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.612498 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.612477    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq47p\" (UniqueName: \"kubernetes.io/projected/14b3a283-8e07-4350-8e92-c0044d151af3-kube-api-access-xq47p\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.612720 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.612520    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-trusted-ca-bundle\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.613244 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.613217    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-service-ca\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.613351 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.613251    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-oauth-serving-cert\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.613535 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.613515    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-console-config\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.613780 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.613760    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-trusted-ca-bundle\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.615112 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.615086    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14b3a283-8e07-4350-8e92-c0044d151af3-console-serving-cert\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.615112 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.615107    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14b3a283-8e07-4350-8e92-c0044d151af3-console-oauth-config\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.624176 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.624157    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq47p\" (UniqueName: \"kubernetes.io/projected/14b3a283-8e07-4350-8e92-c0044d151af3-kube-api-access-xq47p\") pod \"console-695c5b8ff5-hsvqr\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") " pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.708765 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.708708    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:16.984336 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:16.984164    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-695c5b8ff5-hsvqr"]
Sep 11 16:33:16.988056 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:33:16.988023    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14b3a283_8e07_4350_8e92_c0044d151af3.slice/crio-a3367a728b22d19a50438fa730b637132d6748e4797e49ef7a2c8db076d3fc25 WatchSource:0}: Error finding container a3367a728b22d19a50438fa730b637132d6748e4797e49ef7a2c8db076d3fc25: Status 404 returned error can't find the container with id a3367a728b22d19a50438fa730b637132d6748e4797e49ef7a2c8db076d3fc25
Sep 11 16:33:17.838826 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:17.838790    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695c5b8ff5-hsvqr" event={"ID":"14b3a283-8e07-4350-8e92-c0044d151af3","Type":"ContainerStarted","Data":"ff734e49460a2c095ddfe26ab8586e4481387988e09c0a71958739241851c18a"}
Sep 11 16:33:17.839216 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:17.838843    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695c5b8ff5-hsvqr" event={"ID":"14b3a283-8e07-4350-8e92-c0044d151af3","Type":"ContainerStarted","Data":"a3367a728b22d19a50438fa730b637132d6748e4797e49ef7a2c8db076d3fc25"}
Sep 11 16:33:17.872985 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:17.872935    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-695c5b8ff5-hsvqr" podStartSLOduration=1.872919235 podStartE2EDuration="1.872919235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:33:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:33:17.87103843 +0000 UTC m=+216.424208367" watchObservedRunningTime="2026-09-11 16:33:17.872919235 +0000 UTC m=+216.426089162"
Sep 11 16:33:19.394280 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.394245    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 16:33:19.394751 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.394618    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="alertmanager" containerID="cri-o://a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca" gracePeriod=120
Sep 11 16:33:19.394751 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.394655    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="kube-rbac-proxy-web" containerID="cri-o://d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16" gracePeriod=120
Sep 11 16:33:19.394751 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.394699    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="config-reloader" containerID="cri-o://6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd" gracePeriod=120
Sep 11 16:33:19.394751 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.394635    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="kube-rbac-proxy" containerID="cri-o://046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848" gracePeriod=120
Sep 11 16:33:19.394993 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.394665    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="kube-rbac-proxy-metric" containerID="cri-o://62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25" gracePeriod=120
Sep 11 16:33:19.394993 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.394662    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="prom-label-proxy" containerID="cri-o://62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae" gracePeriod=120
Sep 11 16:33:19.849493 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.849463    2545 generic.go:356] "Generic (PLEG): container finished" podID="55643449-6537-415d-b51a-86794285a80a" containerID="62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae" exitCode=0
Sep 11 16:33:19.849493 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.849489    2545 generic.go:356] "Generic (PLEG): container finished" podID="55643449-6537-415d-b51a-86794285a80a" containerID="046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848" exitCode=0
Sep 11 16:33:19.849493 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.849495    2545 generic.go:356] "Generic (PLEG): container finished" podID="55643449-6537-415d-b51a-86794285a80a" containerID="6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd" exitCode=0
Sep 11 16:33:19.849493 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.849503    2545 generic.go:356] "Generic (PLEG): container finished" podID="55643449-6537-415d-b51a-86794285a80a" containerID="a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca" exitCode=0
Sep 11 16:33:19.849745 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.849541    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerDied","Data":"62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae"}
Sep 11 16:33:19.849745 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.849613    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerDied","Data":"046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848"}
Sep 11 16:33:19.849745 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.849641    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerDied","Data":"6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd"}
Sep 11 16:33:19.849745 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:19.849664    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerDied","Data":"a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca"}
Sep 11 16:33:20.653114 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.653091    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:20.744007 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.743984    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy\") pod \"55643449-6537-415d-b51a-86794285a80a\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") "
Sep 11 16:33:20.744156 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.744026    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-main-tls\") pod \"55643449-6537-415d-b51a-86794285a80a\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") "
Sep 11 16:33:20.744156 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.744053    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy-web\") pod \"55643449-6537-415d-b51a-86794285a80a\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") "
Sep 11 16:33:20.744263 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.744172    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55643449-6537-415d-b51a-86794285a80a-alertmanager-main-db\") pod \"55643449-6537-415d-b51a-86794285a80a\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") "
Sep 11 16:33:20.744263 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.744220    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfp7j\" (UniqueName: \"kubernetes.io/projected/55643449-6537-415d-b51a-86794285a80a-kube-api-access-jfp7j\") pod \"55643449-6537-415d-b51a-86794285a80a\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") "
Sep 11 16:33:20.744263 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.744249    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55643449-6537-415d-b51a-86794285a80a-alertmanager-trusted-ca-bundle\") pod \"55643449-6537-415d-b51a-86794285a80a\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") "
Sep 11 16:33:20.744407 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.744284    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"55643449-6537-415d-b51a-86794285a80a\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") "
Sep 11 16:33:20.744694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.744526    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55643449-6537-415d-b51a-86794285a80a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "55643449-6537-415d-b51a-86794285a80a" (UID: "55643449-6537-415d-b51a-86794285a80a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:33:20.744694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.744602    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55643449-6537-415d-b51a-86794285a80a-tls-assets\") pod \"55643449-6537-415d-b51a-86794285a80a\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") "
Sep 11 16:33:20.744694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.744650    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-cluster-tls-config\") pod \"55643449-6537-415d-b51a-86794285a80a\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") "
Sep 11 16:33:20.744914 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.744699    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-web-config\") pod \"55643449-6537-415d-b51a-86794285a80a\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") "
Sep 11 16:33:20.744914 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.744722    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-config-volume\") pod \"55643449-6537-415d-b51a-86794285a80a\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") "
Sep 11 16:33:20.744914 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.744764    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55643449-6537-415d-b51a-86794285a80a-metrics-client-ca\") pod \"55643449-6537-415d-b51a-86794285a80a\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") "
Sep 11 16:33:20.744914 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.744789    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55643449-6537-415d-b51a-86794285a80a-config-out\") pod \"55643449-6537-415d-b51a-86794285a80a\" (UID: \"55643449-6537-415d-b51a-86794285a80a\") "
Sep 11 16:33:20.744914 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.744811    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55643449-6537-415d-b51a-86794285a80a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "55643449-6537-415d-b51a-86794285a80a" (UID: "55643449-6537-415d-b51a-86794285a80a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:33:20.745152 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.745059    2545 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55643449-6537-415d-b51a-86794285a80a-alertmanager-main-db\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:33:20.745152 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.745079    2545 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55643449-6537-415d-b51a-86794285a80a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:33:20.745620 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.745597    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55643449-6537-415d-b51a-86794285a80a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "55643449-6537-415d-b51a-86794285a80a" (UID: "55643449-6537-415d-b51a-86794285a80a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:33:20.747949 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.747919    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "55643449-6537-415d-b51a-86794285a80a" (UID: "55643449-6537-415d-b51a-86794285a80a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:33:20.748070 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.747979    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55643449-6537-415d-b51a-86794285a80a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "55643449-6537-415d-b51a-86794285a80a" (UID: "55643449-6537-415d-b51a-86794285a80a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:33:20.748459 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.748430    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55643449-6537-415d-b51a-86794285a80a-kube-api-access-jfp7j" (OuterVolumeSpecName: "kube-api-access-jfp7j") pod "55643449-6537-415d-b51a-86794285a80a" (UID: "55643449-6537-415d-b51a-86794285a80a"). InnerVolumeSpecName "kube-api-access-jfp7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:33:20.748583 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.748563    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "55643449-6537-415d-b51a-86794285a80a" (UID: "55643449-6537-415d-b51a-86794285a80a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:33:20.748717 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.748692    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "55643449-6537-415d-b51a-86794285a80a" (UID: "55643449-6537-415d-b51a-86794285a80a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:33:20.748857 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.748840    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55643449-6537-415d-b51a-86794285a80a-config-out" (OuterVolumeSpecName: "config-out") pod "55643449-6537-415d-b51a-86794285a80a" (UID: "55643449-6537-415d-b51a-86794285a80a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:33:20.749165 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.749147    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-config-volume" (OuterVolumeSpecName: "config-volume") pod "55643449-6537-415d-b51a-86794285a80a" (UID: "55643449-6537-415d-b51a-86794285a80a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:33:20.752529 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.752502    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "55643449-6537-415d-b51a-86794285a80a" (UID: "55643449-6537-415d-b51a-86794285a80a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:33:20.754792 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.754761    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "55643449-6537-415d-b51a-86794285a80a" (UID: "55643449-6537-415d-b51a-86794285a80a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:33:20.761799 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.761772    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-web-config" (OuterVolumeSpecName: "web-config") pod "55643449-6537-415d-b51a-86794285a80a" (UID: "55643449-6537-415d-b51a-86794285a80a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:33:20.846386 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.846342    2545 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:33:20.846386 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.846377    2545 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55643449-6537-415d-b51a-86794285a80a-tls-assets\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:33:20.846386 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.846386    2545 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-cluster-tls-config\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:33:20.846386 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.846396    2545 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-web-config\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:33:20.846686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.846405    2545 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-config-volume\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:33:20.846686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.846413    2545 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55643449-6537-415d-b51a-86794285a80a-metrics-client-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:33:20.846686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.846420    2545 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55643449-6537-415d-b51a-86794285a80a-config-out\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:33:20.846686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.846429    2545 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:33:20.846686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.846438    2545 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-main-tls\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:33:20.846686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.846451    2545 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55643449-6537-415d-b51a-86794285a80a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:33:20.846686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.846464    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfp7j\" (UniqueName: \"kubernetes.io/projected/55643449-6537-415d-b51a-86794285a80a-kube-api-access-jfp7j\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:33:20.857073 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.857050    2545 generic.go:356] "Generic (PLEG): container finished" podID="55643449-6537-415d-b51a-86794285a80a" containerID="62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25" exitCode=0
Sep 11 16:33:20.857073 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.857071    2545 generic.go:356] "Generic (PLEG): container finished" podID="55643449-6537-415d-b51a-86794285a80a" containerID="d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16" exitCode=0
Sep 11 16:33:20.857188 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.857089    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerDied","Data":"62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25"}
Sep 11 16:33:20.857188 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.857121    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:20.857188 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.857175    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerDied","Data":"d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16"}
Sep 11 16:33:20.857272 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.857203    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55643449-6537-415d-b51a-86794285a80a","Type":"ContainerDied","Data":"1855cce492c0c71a3c0323450bef1dd137cf89806bec3d3537a376e5f10e37e1"}
Sep 11 16:33:20.857272 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.857244    2545 scope.go:117] "RemoveContainer" containerID="62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae"
Sep 11 16:33:20.866110 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.866094    2545 scope.go:117] "RemoveContainer" containerID="62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25"
Sep 11 16:33:20.872858 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.872840    2545 scope.go:117] "RemoveContainer" containerID="046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848"
Sep 11 16:33:20.879239 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.879223    2545 scope.go:117] "RemoveContainer" containerID="d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16"
Sep 11 16:33:20.882614 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.882592    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 16:33:20.886237 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.886221    2545 scope.go:117] "RemoveContainer" containerID="6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd"
Sep 11 16:33:20.888582 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.888559    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 16:33:20.893343 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.893328    2545 scope.go:117] "RemoveContainer" containerID="a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca"
Sep 11 16:33:20.900178 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.900161    2545 scope.go:117] "RemoveContainer" containerID="514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c"
Sep 11 16:33:20.906779 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.906761    2545 scope.go:117] "RemoveContainer" containerID="62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae"
Sep 11 16:33:20.907123 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:33:20.907105    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae\": container with ID starting with 62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae not found: ID does not exist" containerID="62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae"
Sep 11 16:33:20.907197 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.907128    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae"} err="failed to get container status \"62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae\": rpc error: code = NotFound desc = could not find container \"62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae\": container with ID starting with 62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae not found: ID does not exist"
Sep 11 16:33:20.907197 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.907142    2545 scope.go:117] "RemoveContainer" containerID="62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25"
Sep 11 16:33:20.907408 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:33:20.907391    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25\": container with ID starting with 62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25 not found: ID does not exist" containerID="62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25"
Sep 11 16:33:20.907450 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.907414    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25"} err="failed to get container status \"62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25\": rpc error: code = NotFound desc = could not find container \"62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25\": container with ID starting with 62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25 not found: ID does not exist"
Sep 11 16:33:20.907450 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.907429    2545 scope.go:117] "RemoveContainer" containerID="046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848"
Sep 11 16:33:20.907645 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:33:20.907628    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848\": container with ID starting with 046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848 not found: ID does not exist" containerID="046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848"
Sep 11 16:33:20.907682 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.907650    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848"} err="failed to get container status \"046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848\": rpc error: code = NotFound desc = could not find container \"046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848\": container with ID starting with 046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848 not found: ID does not exist"
Sep 11 16:33:20.907682 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.907671    2545 scope.go:117] "RemoveContainer" containerID="d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16"
Sep 11 16:33:20.907950 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:33:20.907935    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16\": container with ID starting with d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16 not found: ID does not exist" containerID="d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16"
Sep 11 16:33:20.907996 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.907953    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16"} err="failed to get container status \"d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16\": rpc error: code = NotFound desc = could not find container \"d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16\": container with ID starting with d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16 not found: ID does not exist"
Sep 11 16:33:20.907996 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.907965    2545 scope.go:117] "RemoveContainer" containerID="6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd"
Sep 11 16:33:20.908191 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:33:20.908177    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd\": container with ID starting with 6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd not found: ID does not exist" containerID="6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd"
Sep 11 16:33:20.908230 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.908193    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd"} err="failed to get container status \"6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd\": rpc error: code = NotFound desc = could not find container \"6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd\": container with ID starting with 6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd not found: ID does not exist"
Sep 11 16:33:20.908230 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.908204    2545 scope.go:117] "RemoveContainer" containerID="a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca"
Sep 11 16:33:20.908396 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:33:20.908383    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca\": container with ID starting with a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca not found: ID does not exist" containerID="a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca"
Sep 11 16:33:20.908434 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.908398    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca"} err="failed to get container status \"a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca\": rpc error: code = NotFound desc = could not find container \"a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca\": container with ID starting with a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca not found: ID does not exist"
Sep 11 16:33:20.908434 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.908408    2545 scope.go:117] "RemoveContainer" containerID="514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c"
Sep 11 16:33:20.908598 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:33:20.908581    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c\": container with ID starting with 514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c not found: ID does not exist" containerID="514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c"
Sep 11 16:33:20.908634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.908605    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c"} err="failed to get container status \"514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c\": rpc error: code = NotFound desc = could not find container \"514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c\": container with ID starting with 514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c not found: ID does not exist"
Sep 11 16:33:20.908634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.908620    2545 scope.go:117] "RemoveContainer" containerID="62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae"
Sep 11 16:33:20.908804 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.908790    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae"} err="failed to get container status \"62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae\": rpc error: code = NotFound desc = could not find container \"62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae\": container with ID starting with 62b0fb73ad038ec1b4bd4ea994bf20560491595085eed84156589f5a8cc154ae not found: ID does not exist"
Sep 11 16:33:20.908841 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.908803    2545 scope.go:117] "RemoveContainer" containerID="62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25"
Sep 11 16:33:20.908994 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.908979    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25"} err="failed to get container status \"62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25\": rpc error: code = NotFound desc = could not find container \"62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25\": container with ID starting with 62f6086096295d6440f31476dc606d2027b87355ec3fda7afa0b05c82af97c25 not found: ID does not exist"
Sep 11 16:33:20.908994 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.908993    2545 scope.go:117] "RemoveContainer" containerID="046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848"
Sep 11 16:33:20.909225 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.909211    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848"} err="failed to get container status \"046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848\": rpc error: code = NotFound desc = could not find container \"046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848\": container with ID starting with 046422d8dd59236ed7d86350579a5eea1909e9246745b3ae5186921c4367d848 not found: ID does not exist"
Sep 11 16:33:20.909225 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.909223    2545 scope.go:117] "RemoveContainer" containerID="d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16"
Sep 11 16:33:20.909392 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.909376    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16"} err="failed to get container status \"d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16\": rpc error: code = NotFound desc = could not find container \"d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16\": container with ID starting with d575538734561154a2f7b3a588a407fc6cc69b90d3a2903c7158641875360f16 not found: ID does not exist"
Sep 11 16:33:20.909435 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.909393    2545 scope.go:117] "RemoveContainer" containerID="6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd"
Sep 11 16:33:20.909564 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.909549    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd"} err="failed to get container status \"6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd\": rpc error: code = NotFound desc = could not find container \"6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd\": container with ID starting with 6f8c47fe2820233596811da91a469c7bb6260272e103d136b89edca28bc00ccd not found: ID does not exist"
Sep 11 16:33:20.909564 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.909563    2545 scope.go:117] "RemoveContainer" containerID="a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca"
Sep 11 16:33:20.909729 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.909715    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca"} err="failed to get container status \"a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca\": rpc error: code = NotFound desc = could not find container \"a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca\": container with ID starting with a92b342194b7b7748460e0c083dfa5666101f32b92feb1005aa1cb747282b6ca not found: ID does not exist"
Sep 11 16:33:20.909729 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.909728    2545 scope.go:117] "RemoveContainer" containerID="514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c"
Sep 11 16:33:20.910004 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.909987    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c"} err="failed to get container status \"514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c\": rpc error: code = NotFound desc = could not find container \"514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c\": container with ID starting with 514de947e7b12ae7e1c5b4283f9c76dff06a76f327df56803c3673e5166af55c not found: ID does not exist"
Sep 11 16:33:20.922059 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922038    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 16:33:20.922306 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922295    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="kube-rbac-proxy-web"
Sep 11 16:33:20.922342 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922308    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="kube-rbac-proxy-web"
Sep 11 16:33:20.922342 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922325    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="config-reloader"
Sep 11 16:33:20.922342 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922331    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="config-reloader"
Sep 11 16:33:20.922342 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922337    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="alertmanager"
Sep 11 16:33:20.922342 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922343    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="alertmanager"
Sep 11 16:33:20.922487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922350    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="prom-label-proxy"
Sep 11 16:33:20.922487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922356    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="prom-label-proxy"
Sep 11 16:33:20.922487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922365    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="kube-rbac-proxy"
Sep 11 16:33:20.922487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922371    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="kube-rbac-proxy"
Sep 11 16:33:20.922487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922377    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="kube-rbac-proxy-metric"
Sep 11 16:33:20.922487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922383    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="kube-rbac-proxy-metric"
Sep 11 16:33:20.922487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922393    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="init-config-reloader"
Sep 11 16:33:20.922487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922398    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="init-config-reloader"
Sep 11 16:33:20.922487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922442    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="alertmanager"
Sep 11 16:33:20.922487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922449    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="kube-rbac-proxy-web"
Sep 11 16:33:20.922487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922454    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="config-reloader"
Sep 11 16:33:20.922487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922459    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="kube-rbac-proxy-metric"
Sep 11 16:33:20.922487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922464    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="prom-label-proxy"
Sep 11 16:33:20.922487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.922471    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="55643449-6537-415d-b51a-86794285a80a" containerName="kube-rbac-proxy"
Sep 11 16:33:20.927422 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.927407    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:20.929992 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.929970    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5c4wr\""
Sep 11 16:33:20.930285 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.930252    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 16:33:20.930285 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.930270    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 16:33:20.930441 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.930308    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 16:33:20.930441 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.930399    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 16:33:20.930539 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.930524    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 16:33:20.930620 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.930600    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 16:33:20.930708 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.930525    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 16:33:20.930708 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.930683    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 16:33:20.937457 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.937435    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 16:33:20.950937 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:20.950907    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 16:33:21.048735 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.048660    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/809e1548-970a-4190-8bd9-90ab402c1e50-config-out\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.048735 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.048696    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsqvf\" (UniqueName: \"kubernetes.io/projected/809e1548-970a-4190-8bd9-90ab402c1e50-kube-api-access-nsqvf\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.048735 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.048723    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/809e1548-970a-4190-8bd9-90ab402c1e50-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.048941 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.048816    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.048941 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.048841    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.048941 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.048891    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-web-config\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.048941 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.048927    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-config-volume\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.049069 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.048943    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.049069 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.048959    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.049069 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.049005    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/809e1548-970a-4190-8bd9-90ab402c1e50-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.049069 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.049039    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/809e1548-970a-4190-8bd9-90ab402c1e50-tls-assets\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.049069 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.049056    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.049211 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.049094    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/809e1548-970a-4190-8bd9-90ab402c1e50-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.150101 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.150074    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.150101 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.150103    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.150257 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.150126    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-web-config\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.150257 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.150151    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-config-volume\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.150257 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.150175    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.150257 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.150200    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.150257 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.150235    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/809e1548-970a-4190-8bd9-90ab402c1e50-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.150454 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.150271    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/809e1548-970a-4190-8bd9-90ab402c1e50-tls-assets\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.150454 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.150294    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.150454 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.150330    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/809e1548-970a-4190-8bd9-90ab402c1e50-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.150454 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.150364    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/809e1548-970a-4190-8bd9-90ab402c1e50-config-out\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.150454 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.150384    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsqvf\" (UniqueName: \"kubernetes.io/projected/809e1548-970a-4190-8bd9-90ab402c1e50-kube-api-access-nsqvf\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.150454 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.150406    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/809e1548-970a-4190-8bd9-90ab402c1e50-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.151394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.151364    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/809e1548-970a-4190-8bd9-90ab402c1e50-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.152222 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.152191    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/809e1548-970a-4190-8bd9-90ab402c1e50-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.153677 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.153471    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.153677 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.153504    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.153677 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.153586    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.153677 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.153628    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.153960 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.153733    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-web-config\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.153960 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.153751    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/809e1548-970a-4190-8bd9-90ab402c1e50-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.153960 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.153763    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-config-volume\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.153960 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.153771    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/809e1548-970a-4190-8bd9-90ab402c1e50-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.154473 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.154452    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/809e1548-970a-4190-8bd9-90ab402c1e50-config-out\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.155607 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.155590    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/809e1548-970a-4190-8bd9-90ab402c1e50-tls-assets\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.165492 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.165471    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsqvf\" (UniqueName: \"kubernetes.io/projected/809e1548-970a-4190-8bd9-90ab402c1e50-kube-api-access-nsqvf\") pod \"alertmanager-main-0\" (UID: \"809e1548-970a-4190-8bd9-90ab402c1e50\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.237520 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.237497    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 16:33:21.521116 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.521087    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 16:33:21.525806 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:33:21.525771    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod809e1548_970a_4190_8bd9_90ab402c1e50.slice/crio-847f0983f00bfda526e9fba2b92eb2e7550eea4d8fe5b32d5fa4745b8f989bdc WatchSource:0}: Error finding container 847f0983f00bfda526e9fba2b92eb2e7550eea4d8fe5b32d5fa4745b8f989bdc: Status 404 returned error can't find the container with id 847f0983f00bfda526e9fba2b92eb2e7550eea4d8fe5b32d5fa4745b8f989bdc
Sep 11 16:33:21.867072 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.867039    2545 generic.go:356] "Generic (PLEG): container finished" podID="809e1548-970a-4190-8bd9-90ab402c1e50" containerID="5dc4a5894716b0f6f904b89f946475905fa128c1c94e349c4a1c930573cd8698" exitCode=0
Sep 11 16:33:21.867516 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.867124    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"809e1548-970a-4190-8bd9-90ab402c1e50","Type":"ContainerDied","Data":"5dc4a5894716b0f6f904b89f946475905fa128c1c94e349c4a1c930573cd8698"}
Sep 11 16:33:21.867516 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.867210    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"809e1548-970a-4190-8bd9-90ab402c1e50","Type":"ContainerStarted","Data":"847f0983f00bfda526e9fba2b92eb2e7550eea4d8fe5b32d5fa4745b8f989bdc"}
Sep 11 16:33:21.938242 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:21.938217    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55643449-6537-415d-b51a-86794285a80a" path="/var/lib/kubelet/pods/55643449-6537-415d-b51a-86794285a80a/volumes"
Sep 11 16:33:22.876630 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:22.876596    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"809e1548-970a-4190-8bd9-90ab402c1e50","Type":"ContainerStarted","Data":"199ac376086e19e8f772a3022cef8a088602874fa367d6ce4e85199545c8b0a6"}
Sep 11 16:33:22.877089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:22.876671    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"809e1548-970a-4190-8bd9-90ab402c1e50","Type":"ContainerStarted","Data":"619968d0bcb7c852207cf6f904b8c00510f5fb81e947fea4f13f6e155d258b10"}
Sep 11 16:33:22.877089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:22.876718    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"809e1548-970a-4190-8bd9-90ab402c1e50","Type":"ContainerStarted","Data":"d7e17248e1f72fabfc24e160ff22bd957236a07754b982ca41a889ea5950e463"}
Sep 11 16:33:22.877089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:22.876760    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"809e1548-970a-4190-8bd9-90ab402c1e50","Type":"ContainerStarted","Data":"231c1c4e545a32e3b64a639206ce645e0bfbcfb20ca531955c21a53baaa34d9d"}
Sep 11 16:33:22.877089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:22.876803    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"809e1548-970a-4190-8bd9-90ab402c1e50","Type":"ContainerStarted","Data":"b7ca07e76f70aed86b7c449bd436d27377407c588983bda6bf5e7c55d0d8df10"}
Sep 11 16:33:22.877089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:22.876841    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"809e1548-970a-4190-8bd9-90ab402c1e50","Type":"ContainerStarted","Data":"b831110dacaf91893613e104798731e23de455ea176054b9b33b26f21d91568d"}
Sep 11 16:33:22.907715 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:22.907671    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.890510527 podStartE2EDuration="2.907654715s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.144188ms" isStatefulPod=false podCreationTimestamp="2026-09-11 16:33:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:33:22.905465952 +0000 UTC m=+221.458635879" watchObservedRunningTime="2026-09-11 16:33:22.907654715 +0000 UTC m=+221.460824642"
Sep 11 16:33:26.709238 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:26.709203    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:26.709613 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:26.709282    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:26.714095 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:26.714071    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:33:26.899478 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:33:26.899452    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:34:36.589307 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:36.589270    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r6mgx"]
Sep 11 16:34:36.592498 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:36.592481    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r6mgx"
Sep 11 16:34:36.594649 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:36.594625    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-9h5kh\""
Sep 11 16:34:36.594790 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:36.594692    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 11 16:34:36.594790 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:36.594753    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:34:36.606465 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:36.606443    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r6mgx"]
Sep 11 16:34:36.697470 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:36.697446    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdshp\" (UniqueName: \"kubernetes.io/projected/d5cd648c-b6c2-411d-ba40-9bfe0cd603a2-kube-api-access-cdshp\") pod \"cert-manager-operator-controller-manager-55c444bf86-r6mgx\" (UID: \"d5cd648c-b6c2-411d-ba40-9bfe0cd603a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r6mgx"
Sep 11 16:34:36.697574 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:36.697480    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5cd648c-b6c2-411d-ba40-9bfe0cd603a2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-r6mgx\" (UID: \"d5cd648c-b6c2-411d-ba40-9bfe0cd603a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r6mgx"
Sep 11 16:34:36.798623 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:36.798597    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdshp\" (UniqueName: \"kubernetes.io/projected/d5cd648c-b6c2-411d-ba40-9bfe0cd603a2-kube-api-access-cdshp\") pod \"cert-manager-operator-controller-manager-55c444bf86-r6mgx\" (UID: \"d5cd648c-b6c2-411d-ba40-9bfe0cd603a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r6mgx"
Sep 11 16:34:36.798720 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:36.798633    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5cd648c-b6c2-411d-ba40-9bfe0cd603a2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-r6mgx\" (UID: \"d5cd648c-b6c2-411d-ba40-9bfe0cd603a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r6mgx"
Sep 11 16:34:36.799002 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:36.798987    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5cd648c-b6c2-411d-ba40-9bfe0cd603a2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-r6mgx\" (UID: \"d5cd648c-b6c2-411d-ba40-9bfe0cd603a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r6mgx"
Sep 11 16:34:36.812191 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:36.812166    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdshp\" (UniqueName: \"kubernetes.io/projected/d5cd648c-b6c2-411d-ba40-9bfe0cd603a2-kube-api-access-cdshp\") pod \"cert-manager-operator-controller-manager-55c444bf86-r6mgx\" (UID: \"d5cd648c-b6c2-411d-ba40-9bfe0cd603a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r6mgx"
Sep 11 16:34:36.901976 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:36.901921    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r6mgx"
Sep 11 16:34:37.182922 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:37.182897    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r6mgx"]
Sep 11 16:34:37.185826 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:34:37.185790    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5cd648c_b6c2_411d_ba40_9bfe0cd603a2.slice/crio-b093f77b3a20a0c8251807b3f3c02917bad3951d6240353e40a4ceb72127b154 WatchSource:0}: Error finding container b093f77b3a20a0c8251807b3f3c02917bad3951d6240353e40a4ceb72127b154: Status 404 returned error can't find the container with id b093f77b3a20a0c8251807b3f3c02917bad3951d6240353e40a4ceb72127b154
Sep 11 16:34:38.178471 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:38.178440    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r6mgx" event={"ID":"d5cd648c-b6c2-411d-ba40-9bfe0cd603a2","Type":"ContainerStarted","Data":"b093f77b3a20a0c8251807b3f3c02917bad3951d6240353e40a4ceb72127b154"}
Sep 11 16:34:41.195414 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:41.195375    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r6mgx" event={"ID":"d5cd648c-b6c2-411d-ba40-9bfe0cd603a2","Type":"ContainerStarted","Data":"f8464f17e230c7a984eec8e72ae6fc0c22d79e6862601c5021555454d2b4f463"}
Sep 11 16:34:41.224101 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:41.224039    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r6mgx" podStartSLOduration=1.983115499 podStartE2EDuration="5.224023662s" totalImagesPullingTime="3.240908163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:34:41.220634639 +0000 UTC m=+299.773804567" watchObservedRunningTime="2026-09-11 16:34:41.224023662 +0000 UTC m=+299.777193590"
Sep 11 16:34:41.870570 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:41.870534    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:34:41.872924 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:41.872901    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:34:41.882851 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:41.882830    2545 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 16:34:43.738430 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:43.738402    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9j4bs"]
Sep 11 16:34:43.741691 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:43.741675    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9j4bs"
Sep 11 16:34:43.743911 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:43.743881    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 16:34:43.744049 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:43.743938    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 16:34:43.744375 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:43.744360    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-4rtjz\""
Sep 11 16:34:43.753378 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:43.753358    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9j4bs"]
Sep 11 16:34:43.856947 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:43.856916    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddpfk\" (UniqueName: \"kubernetes.io/projected/72e74951-b4be-4370-9bdf-eeab207d4dc1-kube-api-access-ddpfk\") pod \"cert-manager-cainjector-559cdd8c87-9j4bs\" (UID: \"72e74951-b4be-4370-9bdf-eeab207d4dc1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9j4bs"
Sep 11 16:34:43.857099 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:43.856959    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72e74951-b4be-4370-9bdf-eeab207d4dc1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9j4bs\" (UID: \"72e74951-b4be-4370-9bdf-eeab207d4dc1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9j4bs"
Sep 11 16:34:43.958124 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:43.958097    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddpfk\" (UniqueName: \"kubernetes.io/projected/72e74951-b4be-4370-9bdf-eeab207d4dc1-kube-api-access-ddpfk\") pod \"cert-manager-cainjector-559cdd8c87-9j4bs\" (UID: \"72e74951-b4be-4370-9bdf-eeab207d4dc1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9j4bs"
Sep 11 16:34:43.958299 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:43.958137    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72e74951-b4be-4370-9bdf-eeab207d4dc1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9j4bs\" (UID: \"72e74951-b4be-4370-9bdf-eeab207d4dc1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9j4bs"
Sep 11 16:34:43.967571 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:43.967540    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72e74951-b4be-4370-9bdf-eeab207d4dc1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9j4bs\" (UID: \"72e74951-b4be-4370-9bdf-eeab207d4dc1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9j4bs"
Sep 11 16:34:43.967571 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:43.967553    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddpfk\" (UniqueName: \"kubernetes.io/projected/72e74951-b4be-4370-9bdf-eeab207d4dc1-kube-api-access-ddpfk\") pod \"cert-manager-cainjector-559cdd8c87-9j4bs\" (UID: \"72e74951-b4be-4370-9bdf-eeab207d4dc1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9j4bs"
Sep 11 16:34:44.061206 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:44.061145    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9j4bs"
Sep 11 16:34:44.338626 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:44.338546    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9j4bs"]
Sep 11 16:34:44.341157 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:34:44.341130    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72e74951_b4be_4370_9bdf_eeab207d4dc1.slice/crio-d12e48f484a519d6e5a93117111c55ef6dc9650cc9c4a81ed831e61ae4ecb718 WatchSource:0}: Error finding container d12e48f484a519d6e5a93117111c55ef6dc9650cc9c4a81ed831e61ae4ecb718: Status 404 returned error can't find the container with id d12e48f484a519d6e5a93117111c55ef6dc9650cc9c4a81ed831e61ae4ecb718
Sep 11 16:34:44.343090 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:44.343070    2545 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 16:34:45.215430 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:45.215389    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9j4bs" event={"ID":"72e74951-b4be-4370-9bdf-eeab207d4dc1","Type":"ContainerStarted","Data":"d12e48f484a519d6e5a93117111c55ef6dc9650cc9c4a81ed831e61ae4ecb718"}
Sep 11 16:34:47.226338 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:47.226252    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9j4bs" event={"ID":"72e74951-b4be-4370-9bdf-eeab207d4dc1","Type":"ContainerStarted","Data":"a9a80f5713838052d93a16e2b9adef2db50d42a3de8cc13f3bcfe77ab1c0aa7b"}
Sep 11 16:34:47.252344 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:47.252292    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9j4bs" podStartSLOduration=1.6617209179999999 podStartE2EDuration="4.252276358s" totalImagesPullingTime="2.59055544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:34:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:34:47.252089968 +0000 UTC m=+305.805259894" watchObservedRunningTime="2026-09-11 16:34:47.252276358 +0000 UTC m=+305.805446285"
Sep 11 16:34:53.734572 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:53.734530    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24qt"]
Sep 11 16:34:53.738034 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:53.738011    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24qt"
Sep 11 16:34:53.740370 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:53.740348    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-82vx6\""
Sep 11 16:34:53.741189 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:53.741176    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 16:34:53.741256 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:53.741219    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:34:53.752147 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:53.752124    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24qt"]
Sep 11 16:34:53.835048 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:53.835023    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/373ff2f4-afd2-4050-a55a-407b14bd4537-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g24qt\" (UID: \"373ff2f4-afd2-4050-a55a-407b14bd4537\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24qt"
Sep 11 16:34:53.835173 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:53.835076    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxxn6\" (UniqueName: \"kubernetes.io/projected/373ff2f4-afd2-4050-a55a-407b14bd4537-kube-api-access-bxxn6\") pod \"openshift-lws-operator-fd8ccff4c-g24qt\" (UID: \"373ff2f4-afd2-4050-a55a-407b14bd4537\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24qt"
Sep 11 16:34:53.935454 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:53.935433    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/373ff2f4-afd2-4050-a55a-407b14bd4537-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g24qt\" (UID: \"373ff2f4-afd2-4050-a55a-407b14bd4537\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24qt"
Sep 11 16:34:53.935587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:53.935489    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxxn6\" (UniqueName: \"kubernetes.io/projected/373ff2f4-afd2-4050-a55a-407b14bd4537-kube-api-access-bxxn6\") pod \"openshift-lws-operator-fd8ccff4c-g24qt\" (UID: \"373ff2f4-afd2-4050-a55a-407b14bd4537\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24qt"
Sep 11 16:34:53.935811 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:53.935792    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/373ff2f4-afd2-4050-a55a-407b14bd4537-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g24qt\" (UID: \"373ff2f4-afd2-4050-a55a-407b14bd4537\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24qt"
Sep 11 16:34:53.944836 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:53.944812    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxxn6\" (UniqueName: \"kubernetes.io/projected/373ff2f4-afd2-4050-a55a-407b14bd4537-kube-api-access-bxxn6\") pod \"openshift-lws-operator-fd8ccff4c-g24qt\" (UID: \"373ff2f4-afd2-4050-a55a-407b14bd4537\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24qt"
Sep 11 16:34:54.046967 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:54.046948    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24qt"
Sep 11 16:34:54.318958 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:54.318888    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24qt"]
Sep 11 16:34:54.323236 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:34:54.323206    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod373ff2f4_afd2_4050_a55a_407b14bd4537.slice/crio-9102f3f69ee9660e93b13c18490ca75eeb404c8abd00b43be08309c908e4006f WatchSource:0}: Error finding container 9102f3f69ee9660e93b13c18490ca75eeb404c8abd00b43be08309c908e4006f: Status 404 returned error can't find the container with id 9102f3f69ee9660e93b13c18490ca75eeb404c8abd00b43be08309c908e4006f
Sep 11 16:34:55.267217 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:55.267182    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24qt" event={"ID":"373ff2f4-afd2-4050-a55a-407b14bd4537","Type":"ContainerStarted","Data":"9102f3f69ee9660e93b13c18490ca75eeb404c8abd00b43be08309c908e4006f"}
Sep 11 16:34:57.279058 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:57.279023    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24qt" event={"ID":"373ff2f4-afd2-4050-a55a-407b14bd4537","Type":"ContainerStarted","Data":"420bb3bcb54ced0592b1a5d404cd051c264147c880bd49544be712367e4e3554"}
Sep 11 16:34:57.296305 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:34:57.296247    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24qt" podStartSLOduration=2.097221049 podStartE2EDuration="4.296230644s" totalImagesPullingTime="2.199009595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:34:57.296075601 +0000 UTC m=+315.849245530" watchObservedRunningTime="2026-09-11 16:34:57.296230644 +0000 UTC m=+315.849400570"
Sep 11 16:35:16.506586 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.506551    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"]
Sep 11 16:35:16.513070 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.513052    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.515124 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.515095    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 16:35:16.515503 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.515487    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 16:35:16.515814 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.515793    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mvggq\""
Sep 11 16:35:16.516000 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.515984    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 16:35:16.516319 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.516307    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 16:35:16.527350 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.527328    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"]
Sep 11 16:35:16.605948 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.605908    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzwpx\" (UniqueName: \"kubernetes.io/projected/f2a8594c-3871-41a0-b96f-73b0c0175bfa-kube-api-access-gzwpx\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.605948 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.605958    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2a8594c-3871-41a0-b96f-73b0c0175bfa-webhook-cert\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.606158 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.605983    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2a8594c-3871-41a0-b96f-73b0c0175bfa-manifests\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.606158 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.606032    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2a8594c-3871-41a0-b96f-73b0c0175bfa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.606158 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.606062    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2a8594c-3871-41a0-b96f-73b0c0175bfa-tmp\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.707306 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.707274    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2a8594c-3871-41a0-b96f-73b0c0175bfa-tmp\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.707463 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.707330    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzwpx\" (UniqueName: \"kubernetes.io/projected/f2a8594c-3871-41a0-b96f-73b0c0175bfa-kube-api-access-gzwpx\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.707463 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.707355    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2a8594c-3871-41a0-b96f-73b0c0175bfa-webhook-cert\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.707582 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.707461    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2a8594c-3871-41a0-b96f-73b0c0175bfa-manifests\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.707582 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.707502    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2a8594c-3871-41a0-b96f-73b0c0175bfa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.707820 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.707793    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2a8594c-3871-41a0-b96f-73b0c0175bfa-manifests\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.709903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.709845    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2a8594c-3871-41a0-b96f-73b0c0175bfa-tmp\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.710026 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.710009    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2a8594c-3871-41a0-b96f-73b0c0175bfa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.710086 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.710073    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2a8594c-3871-41a0-b96f-73b0c0175bfa-webhook-cert\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.720674 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.720654    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzwpx\" (UniqueName: \"kubernetes.io/projected/f2a8594c-3871-41a0-b96f-73b0c0175bfa-kube-api-access-gzwpx\") pod \"opendatahub-operator-controller-manager-665d69bffd-dpbh5\" (UID: \"f2a8594c-3871-41a0-b96f-73b0c0175bfa\") " pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:16.823192 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:16.823112    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:17.114070 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:17.113977    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"]
Sep 11 16:35:17.118113 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:35:17.118084    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a8594c_3871_41a0_b96f_73b0c0175bfa.slice/crio-e89808cd4184d540951631b617e0b496f47817a8963c66731fda0d564f369f02 WatchSource:0}: Error finding container e89808cd4184d540951631b617e0b496f47817a8963c66731fda0d564f369f02: Status 404 returned error can't find the container with id e89808cd4184d540951631b617e0b496f47817a8963c66731fda0d564f369f02
Sep 11 16:35:17.368923 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:17.368838    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5" event={"ID":"f2a8594c-3871-41a0-b96f-73b0c0175bfa","Type":"ContainerStarted","Data":"e89808cd4184d540951631b617e0b496f47817a8963c66731fda0d564f369f02"}
Sep 11 16:35:20.386585 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:20.386508    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5" event={"ID":"f2a8594c-3871-41a0-b96f-73b0c0175bfa","Type":"ContainerStarted","Data":"c516842a90797ee038f08dc5c4b45c47b8f12bc9cee99019477a008386d5b775"}
Sep 11 16:35:21.392931 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:21.392887    2545 generic.go:356] "Generic (PLEG): container finished" podID="f2a8594c-3871-41a0-b96f-73b0c0175bfa" containerID="c516842a90797ee038f08dc5c4b45c47b8f12bc9cee99019477a008386d5b775" exitCode=0
Sep 11 16:35:21.393402 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:21.392948    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5" event={"ID":"f2a8594c-3871-41a0-b96f-73b0c0175bfa","Type":"ContainerDied","Data":"c516842a90797ee038f08dc5c4b45c47b8f12bc9cee99019477a008386d5b775"}
Sep 11 16:35:22.400439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:22.400407    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5" event={"ID":"f2a8594c-3871-41a0-b96f-73b0c0175bfa","Type":"ContainerStarted","Data":"b8a37d89cc4f04de1a47ea480de358ec879fc1104a4e504062b39763e3997a56"}
Sep 11 16:35:22.400936 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:22.400510    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:22.432219 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:22.432172    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5" podStartSLOduration=2.082386867 podStartE2EDuration="6.432157263s" totalImagesPullingTime="3.298669937s" totalInitContainerRuntime="1.051100459s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:35:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:35:22.432122786 +0000 UTC m=+340.985292715" watchObservedRunningTime="2026-09-11 16:35:22.432157263 +0000 UTC m=+340.985327189"
Sep 11 16:35:33.408129 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:33.408045    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-665d69bffd-dpbh5"
Sep 11 16:35:43.219723 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.219681    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"]
Sep 11 16:35:43.224632 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.224607    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.227584 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.227562    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-2h9hv\""
Sep 11 16:35:43.227686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.227636    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 16:35:43.227686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.227651    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 16:35:43.227962 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.227947    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 16:35:43.239404 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.239374    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"]
Sep 11 16:35:43.324215 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.324193    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.324311 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.324227    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92d8390b-90e0-4d31-ac76-b6496a647f85-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.324311 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.324260    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.324311 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.324282    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92d8390b-90e0-4d31-ac76-b6496a647f85-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.324440 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.324315    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.324440 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.324334    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.324440 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.324348    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.324533 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.324431    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92d8390b-90e0-4d31-ac76-b6496a647f85-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.324533 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.324462    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2jlq\" (UniqueName: \"kubernetes.io/projected/92d8390b-90e0-4d31-ac76-b6496a647f85-kube-api-access-f2jlq\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.425463 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.425435    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.425577 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.425467    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.425577 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.425504    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92d8390b-90e0-4d31-ac76-b6496a647f85-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.425577 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.425529    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2jlq\" (UniqueName: \"kubernetes.io/projected/92d8390b-90e0-4d31-ac76-b6496a647f85-kube-api-access-f2jlq\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.425793 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.425585    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.425793 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.425620    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92d8390b-90e0-4d31-ac76-b6496a647f85-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.425793 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.425681    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.425793 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.425712    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92d8390b-90e0-4d31-ac76-b6496a647f85-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.425793 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.425751    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.426114 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.425838    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.426114 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.425951    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.426212 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.426139    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.426212 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.426163    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.426686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.426665    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92d8390b-90e0-4d31-ac76-b6496a647f85-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.428036 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.428016    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92d8390b-90e0-4d31-ac76-b6496a647f85-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.428200 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.428178    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92d8390b-90e0-4d31-ac76-b6496a647f85-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.442829 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.442805    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2jlq\" (UniqueName: \"kubernetes.io/projected/92d8390b-90e0-4d31-ac76-b6496a647f85-kube-api-access-f2jlq\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.444631 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.444613    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92d8390b-90e0-4d31-ac76-b6496a647f85-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p\" (UID: \"92d8390b-90e0-4d31-ac76-b6496a647f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.534886 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.534852    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:43.812656 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:43.812505    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"]
Sep 11 16:35:43.814750 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:35:43.814727    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d8390b_90e0_4d31_ac76_b6496a647f85.slice/crio-37f41edf19a03df461db11c2af08133b9cce4d7da7b1b5feb341a61d1a99959c WatchSource:0}: Error finding container 37f41edf19a03df461db11c2af08133b9cce4d7da7b1b5feb341a61d1a99959c: Status 404 returned error can't find the container with id 37f41edf19a03df461db11c2af08133b9cce4d7da7b1b5feb341a61d1a99959c
Sep 11 16:35:44.503087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:44.503053    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p" event={"ID":"92d8390b-90e0-4d31-ac76-b6496a647f85","Type":"ContainerStarted","Data":"37f41edf19a03df461db11c2af08133b9cce4d7da7b1b5feb341a61d1a99959c"}
Sep 11 16:35:46.195338 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.195303    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-trq6d"]
Sep 11 16:35:46.202705 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.202684    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-trq6d"
Sep 11 16:35:46.205312 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.205291    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 11 16:35:46.205444 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.205294    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-29rtn\""
Sep 11 16:35:46.209748 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.209730    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-trq6d"]
Sep 11 16:35:46.244904 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.244879    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6p2q\" (UniqueName: \"kubernetes.io/projected/381b5264-ef15-4c99-9e2c-924f70fed40f-kube-api-access-d6p2q\") pod \"odh-model-controller-7998c67dc4-trq6d\" (UID: \"381b5264-ef15-4c99-9e2c-924f70fed40f\") " pod="opendatahub/odh-model-controller-7998c67dc4-trq6d"
Sep 11 16:35:46.245022 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.244922    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/381b5264-ef15-4c99-9e2c-924f70fed40f-cert\") pod \"odh-model-controller-7998c67dc4-trq6d\" (UID: \"381b5264-ef15-4c99-9e2c-924f70fed40f\") " pod="opendatahub/odh-model-controller-7998c67dc4-trq6d"
Sep 11 16:35:46.345544 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.345507    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/381b5264-ef15-4c99-9e2c-924f70fed40f-cert\") pod \"odh-model-controller-7998c67dc4-trq6d\" (UID: \"381b5264-ef15-4c99-9e2c-924f70fed40f\") " pod="opendatahub/odh-model-controller-7998c67dc4-trq6d"
Sep 11 16:35:46.345709 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.345641    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6p2q\" (UniqueName: \"kubernetes.io/projected/381b5264-ef15-4c99-9e2c-924f70fed40f-kube-api-access-d6p2q\") pod \"odh-model-controller-7998c67dc4-trq6d\" (UID: \"381b5264-ef15-4c99-9e2c-924f70fed40f\") " pod="opendatahub/odh-model-controller-7998c67dc4-trq6d"
Sep 11 16:35:46.345709 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:35:46.345676    2545 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 11 16:35:46.345823 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:35:46.345761    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/381b5264-ef15-4c99-9e2c-924f70fed40f-cert podName:381b5264-ef15-4c99-9e2c-924f70fed40f nodeName:}" failed. No retries permitted until 2026-09-11 16:35:46.845731347 +0000 UTC m=+365.398901253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/381b5264-ef15-4c99-9e2c-924f70fed40f-cert") pod "odh-model-controller-7998c67dc4-trq6d" (UID: "381b5264-ef15-4c99-9e2c-924f70fed40f") : secret "odh-model-controller-webhook-cert" not found
Sep 11 16:35:46.357642 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.357617    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6p2q\" (UniqueName: \"kubernetes.io/projected/381b5264-ef15-4c99-9e2c-924f70fed40f-kube-api-access-d6p2q\") pod \"odh-model-controller-7998c67dc4-trq6d\" (UID: \"381b5264-ef15-4c99-9e2c-924f70fed40f\") " pod="opendatahub/odh-model-controller-7998c67dc4-trq6d"
Sep 11 16:35:46.669403 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.669358    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:35:46.669515 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.669436    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:35:46.669515 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.669466    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:35:46.848687 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.848635    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/381b5264-ef15-4c99-9e2c-924f70fed40f-cert\") pod \"odh-model-controller-7998c67dc4-trq6d\" (UID: \"381b5264-ef15-4c99-9e2c-924f70fed40f\") " pod="opendatahub/odh-model-controller-7998c67dc4-trq6d"
Sep 11 16:35:46.851179 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.851135    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/381b5264-ef15-4c99-9e2c-924f70fed40f-cert\") pod \"odh-model-controller-7998c67dc4-trq6d\" (UID: \"381b5264-ef15-4c99-9e2c-924f70fed40f\") " pod="opendatahub/odh-model-controller-7998c67dc4-trq6d"
Sep 11 16:35:46.935284 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.935207    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-j98b2"]
Sep 11 16:35:46.938601 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.938585    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-j98b2"
Sep 11 16:35:46.941415 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.941392    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 11 16:35:46.949216 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.949191    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmvwc\" (UniqueName: \"kubernetes.io/projected/10039705-5220-481d-8445-a97e9eb6285d-kube-api-access-jmvwc\") pod \"model-serving-api-54d89bd5fc-j98b2\" (UID: \"10039705-5220-481d-8445-a97e9eb6285d\") " pod="opendatahub/model-serving-api-54d89bd5fc-j98b2"
Sep 11 16:35:46.949323 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.949291    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10039705-5220-481d-8445-a97e9eb6285d-tls-certs\") pod \"model-serving-api-54d89bd5fc-j98b2\" (UID: \"10039705-5220-481d-8445-a97e9eb6285d\") " pod="opendatahub/model-serving-api-54d89bd5fc-j98b2"
Sep 11 16:35:46.964638 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:46.964617    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-j98b2"]
Sep 11 16:35:47.050164 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:47.050131    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10039705-5220-481d-8445-a97e9eb6285d-tls-certs\") pod \"model-serving-api-54d89bd5fc-j98b2\" (UID: \"10039705-5220-481d-8445-a97e9eb6285d\") " pod="opendatahub/model-serving-api-54d89bd5fc-j98b2"
Sep 11 16:35:47.050297 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:47.050213    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmvwc\" (UniqueName: \"kubernetes.io/projected/10039705-5220-481d-8445-a97e9eb6285d-kube-api-access-jmvwc\") pod \"model-serving-api-54d89bd5fc-j98b2\" (UID: \"10039705-5220-481d-8445-a97e9eb6285d\") " pod="opendatahub/model-serving-api-54d89bd5fc-j98b2"
Sep 11 16:35:47.050297 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:35:47.050282    2545 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 11 16:35:47.050366 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:35:47.050351    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10039705-5220-481d-8445-a97e9eb6285d-tls-certs podName:10039705-5220-481d-8445-a97e9eb6285d nodeName:}" failed. No retries permitted until 2026-09-11 16:35:47.550332813 +0000 UTC m=+366.103502722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/10039705-5220-481d-8445-a97e9eb6285d-tls-certs") pod "model-serving-api-54d89bd5fc-j98b2" (UID: "10039705-5220-481d-8445-a97e9eb6285d") : secret "model-serving-api-tls" not found
Sep 11 16:35:47.075895 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:47.075845    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmvwc\" (UniqueName: \"kubernetes.io/projected/10039705-5220-481d-8445-a97e9eb6285d-kube-api-access-jmvwc\") pod \"model-serving-api-54d89bd5fc-j98b2\" (UID: \"10039705-5220-481d-8445-a97e9eb6285d\") " pod="opendatahub/model-serving-api-54d89bd5fc-j98b2"
Sep 11 16:35:47.117687 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:47.117662    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-trq6d"
Sep 11 16:35:47.403234 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:47.403208    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-trq6d"]
Sep 11 16:35:47.405724 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:35:47.405690    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod381b5264_ef15_4c99_9e2c_924f70fed40f.slice/crio-5a13aec9240c646fe3c220d289e372e405cd00fd31763c6fe52ab7e8f7333754 WatchSource:0}: Error finding container 5a13aec9240c646fe3c220d289e372e405cd00fd31763c6fe52ab7e8f7333754: Status 404 returned error can't find the container with id 5a13aec9240c646fe3c220d289e372e405cd00fd31763c6fe52ab7e8f7333754
Sep 11 16:35:47.525774 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:47.525745    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-trq6d" event={"ID":"381b5264-ef15-4c99-9e2c-924f70fed40f","Type":"ContainerStarted","Data":"5a13aec9240c646fe3c220d289e372e405cd00fd31763c6fe52ab7e8f7333754"}
Sep 11 16:35:47.527200 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:47.527171    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p" event={"ID":"92d8390b-90e0-4d31-ac76-b6496a647f85","Type":"ContainerStarted","Data":"0a858753d6c2c2ca770a7777c897fdb0e66dc3349f8929dc8236a7b4d8cc3838"}
Sep 11 16:35:47.535488 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:47.535470    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:47.539880 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:47.539845    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:47.554082 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:47.554058    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10039705-5220-481d-8445-a97e9eb6285d-tls-certs\") pod \"model-serving-api-54d89bd5fc-j98b2\" (UID: \"10039705-5220-481d-8445-a97e9eb6285d\") " pod="opendatahub/model-serving-api-54d89bd5fc-j98b2"
Sep 11 16:35:47.555125 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:47.555077    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p" podStartSLOduration=1.702818003 podStartE2EDuration="4.554987962s" totalImagesPullingTime="2.852169959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:35:47.553799039 +0000 UTC m=+366.106968988" watchObservedRunningTime="2026-09-11 16:35:47.554987962 +0000 UTC m=+366.108157889"
Sep 11 16:35:47.556661 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:47.556643    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10039705-5220-481d-8445-a97e9eb6285d-tls-certs\") pod \"model-serving-api-54d89bd5fc-j98b2\" (UID: \"10039705-5220-481d-8445-a97e9eb6285d\") " pod="opendatahub/model-serving-api-54d89bd5fc-j98b2"
Sep 11 16:35:47.849055 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:47.849024    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-j98b2"
Sep 11 16:35:48.125037 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:48.124958    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-j98b2"]
Sep 11 16:35:48.128840 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:35:48.128808    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10039705_5220_481d_8445_a97e9eb6285d.slice/crio-d8f5cc9c38bd88ccc00d60841b7775940441f79c748293496b9922b4e7df8e3d WatchSource:0}: Error finding container d8f5cc9c38bd88ccc00d60841b7775940441f79c748293496b9922b4e7df8e3d: Status 404 returned error can't find the container with id d8f5cc9c38bd88ccc00d60841b7775940441f79c748293496b9922b4e7df8e3d
Sep 11 16:35:48.533398 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:48.533365    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-j98b2" event={"ID":"10039705-5220-481d-8445-a97e9eb6285d","Type":"ContainerStarted","Data":"d8f5cc9c38bd88ccc00d60841b7775940441f79c748293496b9922b4e7df8e3d"}
Sep 11 16:35:48.533855 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:48.533693    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:48.534831 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:48.534802    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p"
Sep 11 16:35:51.550614 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:51.550581    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-trq6d" event={"ID":"381b5264-ef15-4c99-9e2c-924f70fed40f","Type":"ContainerStarted","Data":"aa15e6b5177781e9fcacf842e59e8825bf7189e4aadf42bee3c27e009447f794"}
Sep 11 16:35:51.551067 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:51.550773    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-trq6d"
Sep 11 16:35:51.552236 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:51.552214    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-j98b2" event={"ID":"10039705-5220-481d-8445-a97e9eb6285d","Type":"ContainerStarted","Data":"cf89271cd529b3bc0f508313ff2c68a6ab6b5de6c71bf4458ee4c880449a0d3d"}
Sep 11 16:35:51.552357 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:51.552346    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-j98b2"
Sep 11 16:35:51.571231 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:51.571186    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-trq6d" podStartSLOduration=1.889708933 podStartE2EDuration="5.571171697s" totalImagesPullingTime="3.681462764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:35:51.570333629 +0000 UTC m=+370.123503574" watchObservedRunningTime="2026-09-11 16:35:51.571171697 +0000 UTC m=+370.124341621"
Sep 11 16:35:51.610294 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:51.610239    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-j98b2" podStartSLOduration=2.614437265 podStartE2EDuration="5.610223695s" totalImagesPullingTime="2.99578643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:35:51.609021148 +0000 UTC m=+370.162191072" watchObservedRunningTime="2026-09-11 16:35:51.610223695 +0000 UTC m=+370.163393621"
Sep 11 16:35:55.184697 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.184662    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"]
Sep 11 16:35:55.191365 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.191344    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:55.194748 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.194707    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 16:35:55.194927 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.194767    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 16:35:55.194927 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.194779    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 16:35:55.194927 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.194767    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-gw54b\""
Sep 11 16:35:55.203724 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.203706    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"]
Sep 11 16:35:55.217518 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.217493    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb4c0c97-24db-4e12-8ae0-1585f4cd7df1-cert\") pod \"lws-controller-manager-5bd998d9c-9t57f\" (UID: \"eb4c0c97-24db-4e12-8ae0-1585f4cd7df1\") " pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:55.217646 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.217603    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb4c0c97-24db-4e12-8ae0-1585f4cd7df1-metrics-cert\") pod \"lws-controller-manager-5bd998d9c-9t57f\" (UID: \"eb4c0c97-24db-4e12-8ae0-1585f4cd7df1\") " pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:55.217646 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.217636    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eb4c0c97-24db-4e12-8ae0-1585f4cd7df1-manager-config\") pod \"lws-controller-manager-5bd998d9c-9t57f\" (UID: \"eb4c0c97-24db-4e12-8ae0-1585f4cd7df1\") " pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:55.217715 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.217663    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx9lr\" (UniqueName: \"kubernetes.io/projected/eb4c0c97-24db-4e12-8ae0-1585f4cd7df1-kube-api-access-gx9lr\") pod \"lws-controller-manager-5bd998d9c-9t57f\" (UID: \"eb4c0c97-24db-4e12-8ae0-1585f4cd7df1\") " pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:55.318375 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.318340    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb4c0c97-24db-4e12-8ae0-1585f4cd7df1-metrics-cert\") pod \"lws-controller-manager-5bd998d9c-9t57f\" (UID: \"eb4c0c97-24db-4e12-8ae0-1585f4cd7df1\") " pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:55.318563 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.318384    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eb4c0c97-24db-4e12-8ae0-1585f4cd7df1-manager-config\") pod \"lws-controller-manager-5bd998d9c-9t57f\" (UID: \"eb4c0c97-24db-4e12-8ae0-1585f4cd7df1\") " pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:55.318563 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.318408    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx9lr\" (UniqueName: \"kubernetes.io/projected/eb4c0c97-24db-4e12-8ae0-1585f4cd7df1-kube-api-access-gx9lr\") pod \"lws-controller-manager-5bd998d9c-9t57f\" (UID: \"eb4c0c97-24db-4e12-8ae0-1585f4cd7df1\") " pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:55.318563 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.318444    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb4c0c97-24db-4e12-8ae0-1585f4cd7df1-cert\") pod \"lws-controller-manager-5bd998d9c-9t57f\" (UID: \"eb4c0c97-24db-4e12-8ae0-1585f4cd7df1\") " pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:55.319081 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.319044    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eb4c0c97-24db-4e12-8ae0-1585f4cd7df1-manager-config\") pod \"lws-controller-manager-5bd998d9c-9t57f\" (UID: \"eb4c0c97-24db-4e12-8ae0-1585f4cd7df1\") " pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:55.321013 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.320989    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb4c0c97-24db-4e12-8ae0-1585f4cd7df1-metrics-cert\") pod \"lws-controller-manager-5bd998d9c-9t57f\" (UID: \"eb4c0c97-24db-4e12-8ae0-1585f4cd7df1\") " pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:55.321013 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.321004    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb4c0c97-24db-4e12-8ae0-1585f4cd7df1-cert\") pod \"lws-controller-manager-5bd998d9c-9t57f\" (UID: \"eb4c0c97-24db-4e12-8ae0-1585f4cd7df1\") " pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:55.336231 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.336203    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx9lr\" (UniqueName: \"kubernetes.io/projected/eb4c0c97-24db-4e12-8ae0-1585f4cd7df1-kube-api-access-gx9lr\") pod \"lws-controller-manager-5bd998d9c-9t57f\" (UID: \"eb4c0c97-24db-4e12-8ae0-1585f4cd7df1\") " pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:55.500839 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.500815    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:55.784542 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:55.784519    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"]
Sep 11 16:35:55.786517 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:35:55.786489    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4c0c97_24db_4e12_8ae0_1585f4cd7df1.slice/crio-43bbd08d8b1b822c9acd96adff937aa29703e3bd2def913ea8138898956452b6 WatchSource:0}: Error finding container 43bbd08d8b1b822c9acd96adff937aa29703e3bd2def913ea8138898956452b6: Status 404 returned error can't find the container with id 43bbd08d8b1b822c9acd96adff937aa29703e3bd2def913ea8138898956452b6
Sep 11 16:35:56.580092 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:56.580056    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f" event={"ID":"eb4c0c97-24db-4e12-8ae0-1585f4cd7df1","Type":"ContainerStarted","Data":"43bbd08d8b1b822c9acd96adff937aa29703e3bd2def913ea8138898956452b6"}
Sep 11 16:35:57.586367 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:57.586276    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f" event={"ID":"eb4c0c97-24db-4e12-8ae0-1585f4cd7df1","Type":"ContainerStarted","Data":"a95017cc2e2b79d45b9421b7471068245110a77364a3d696c1d4e5c6dc497e68"}
Sep 11 16:35:57.586721 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:57.586393    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:35:57.739468 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:57.739405    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f" podStartSLOduration=1.211230524 podStartE2EDuration="2.73938444s" totalImagesPullingTime="1.528153916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:35:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:35:57.670438634 +0000 UTC m=+376.223608571" watchObservedRunningTime="2026-09-11 16:35:57.73938444 +0000 UTC m=+376.292554367"
Sep 11 16:35:57.834452 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:57.834420    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bhszh"]
Sep 11 16:35:57.837965 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:57.837924    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bhszh"
Sep 11 16:35:57.840457 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:57.840435    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 11 16:35:57.840576 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:57.840544    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-fjbgs\""
Sep 11 16:35:57.853640 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:57.853605    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bhszh"]
Sep 11 16:35:57.943741 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:57.943714    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v8k9\" (UniqueName: \"kubernetes.io/projected/47711732-e59c-4fe5-8650-f8f97b126a14-kube-api-access-4v8k9\") pod \"kserve-controller-manager-d6f4b65b8-bhszh\" (UID: \"47711732-e59c-4fe5-8650-f8f97b126a14\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bhszh"
Sep 11 16:35:57.943906 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:57.943789    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47711732-e59c-4fe5-8650-f8f97b126a14-cert\") pod \"kserve-controller-manager-d6f4b65b8-bhszh\" (UID: \"47711732-e59c-4fe5-8650-f8f97b126a14\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bhszh"
Sep 11 16:35:58.045055 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:58.045024    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v8k9\" (UniqueName: \"kubernetes.io/projected/47711732-e59c-4fe5-8650-f8f97b126a14-kube-api-access-4v8k9\") pod \"kserve-controller-manager-d6f4b65b8-bhszh\" (UID: \"47711732-e59c-4fe5-8650-f8f97b126a14\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bhszh"
Sep 11 16:35:58.045182 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:58.045099    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47711732-e59c-4fe5-8650-f8f97b126a14-cert\") pod \"kserve-controller-manager-d6f4b65b8-bhszh\" (UID: \"47711732-e59c-4fe5-8650-f8f97b126a14\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bhszh"
Sep 11 16:35:58.047687 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:58.047663    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47711732-e59c-4fe5-8650-f8f97b126a14-cert\") pod \"kserve-controller-manager-d6f4b65b8-bhszh\" (UID: \"47711732-e59c-4fe5-8650-f8f97b126a14\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bhszh"
Sep 11 16:35:58.074214 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:58.074183    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v8k9\" (UniqueName: \"kubernetes.io/projected/47711732-e59c-4fe5-8650-f8f97b126a14-kube-api-access-4v8k9\") pod \"kserve-controller-manager-d6f4b65b8-bhszh\" (UID: \"47711732-e59c-4fe5-8650-f8f97b126a14\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bhszh"
Sep 11 16:35:58.148298 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:58.148244    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bhszh"
Sep 11 16:35:58.468010 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:58.467982    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bhszh"]
Sep 11 16:35:58.468175 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:35:58.468155    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47711732_e59c_4fe5_8650_f8f97b126a14.slice/crio-c73b989b6a23c167dfa4cc7d1f2c9af5929b5250c73299db0ac11949d786f63b WatchSource:0}: Error finding container c73b989b6a23c167dfa4cc7d1f2c9af5929b5250c73299db0ac11949d786f63b: Status 404 returned error can't find the container with id c73b989b6a23c167dfa4cc7d1f2c9af5929b5250c73299db0ac11949d786f63b
Sep 11 16:35:58.592351 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:35:58.592317    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bhszh" event={"ID":"47711732-e59c-4fe5-8650-f8f97b126a14","Type":"ContainerStarted","Data":"c73b989b6a23c167dfa4cc7d1f2c9af5929b5250c73299db0ac11949d786f63b"}
Sep 11 16:36:00.605096 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:36:00.605061    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bhszh" event={"ID":"47711732-e59c-4fe5-8650-f8f97b126a14","Type":"ContainerStarted","Data":"ed07b9f026f144fb8dc06deff1d51f0d23989cb87ae311db72f4b2980a491c16"}
Sep 11 16:36:00.605521 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:36:00.605252    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bhszh"
Sep 11 16:36:02.561081 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:36:02.561056    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-j98b2"
Sep 11 16:36:02.575308 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:36:02.575284    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-trq6d"
Sep 11 16:36:02.612084 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:36:02.612040    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bhszh" podStartSLOduration=3.668065888 podStartE2EDuration="5.612022077s" totalImagesPullingTime="1.943956189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:35:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:36:00.64463598 +0000 UTC m=+379.197805922" watchObservedRunningTime="2026-09-11 16:36:02.612022077 +0000 UTC m=+381.165192002"
Sep 11 16:36:08.594339 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:36:08.594312    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5bd998d9c-9t57f"
Sep 11 16:36:31.615216 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:36:31.615182    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bhszh"
Sep 11 16:37:12.872656 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:12.872582    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-695c5b8ff5-hsvqr"]
Sep 11 16:37:37.892387 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:37.892325    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-695c5b8ff5-hsvqr" podUID="14b3a283-8e07-4350-8e92-c0044d151af3" containerName="console" containerID="cri-o://ff734e49460a2c095ddfe26ab8586e4481387988e09c0a71958739241851c18a" gracePeriod=15
Sep 11 16:37:38.066369 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.066340    2545 generic.go:356] "Generic (PLEG): container finished" podID="14b3a283-8e07-4350-8e92-c0044d151af3" containerID="ff734e49460a2c095ddfe26ab8586e4481387988e09c0a71958739241851c18a" exitCode=0
Sep 11 16:37:38.066515 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.066409    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695c5b8ff5-hsvqr" event={"ID":"14b3a283-8e07-4350-8e92-c0044d151af3","Type":"ContainerDied","Data":"ff734e49460a2c095ddfe26ab8586e4481387988e09c0a71958739241851c18a"}
Sep 11 16:37:38.142612 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.142560    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:37:38.171553 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.171528    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-service-ca\") pod \"14b3a283-8e07-4350-8e92-c0044d151af3\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") "
Sep 11 16:37:38.171553 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.171558    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14b3a283-8e07-4350-8e92-c0044d151af3-console-serving-cert\") pod \"14b3a283-8e07-4350-8e92-c0044d151af3\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") "
Sep 11 16:37:38.171778 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.171584    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-oauth-serving-cert\") pod \"14b3a283-8e07-4350-8e92-c0044d151af3\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") "
Sep 11 16:37:38.171778 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.171603    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-console-config\") pod \"14b3a283-8e07-4350-8e92-c0044d151af3\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") "
Sep 11 16:37:38.171778 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.171625    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq47p\" (UniqueName: \"kubernetes.io/projected/14b3a283-8e07-4350-8e92-c0044d151af3-kube-api-access-xq47p\") pod \"14b3a283-8e07-4350-8e92-c0044d151af3\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") "
Sep 11 16:37:38.171778 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.171708    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14b3a283-8e07-4350-8e92-c0044d151af3-console-oauth-config\") pod \"14b3a283-8e07-4350-8e92-c0044d151af3\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") "
Sep 11 16:37:38.171778 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.171737    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-trusted-ca-bundle\") pod \"14b3a283-8e07-4350-8e92-c0044d151af3\" (UID: \"14b3a283-8e07-4350-8e92-c0044d151af3\") "
Sep 11 16:37:38.172129 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.172090    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-service-ca" (OuterVolumeSpecName: "service-ca") pod "14b3a283-8e07-4350-8e92-c0044d151af3" (UID: "14b3a283-8e07-4350-8e92-c0044d151af3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:37:38.172232 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.172131    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "14b3a283-8e07-4350-8e92-c0044d151af3" (UID: "14b3a283-8e07-4350-8e92-c0044d151af3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:37:38.172296 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.172258    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-console-config" (OuterVolumeSpecName: "console-config") pod "14b3a283-8e07-4350-8e92-c0044d151af3" (UID: "14b3a283-8e07-4350-8e92-c0044d151af3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:37:38.172704 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.172663    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "14b3a283-8e07-4350-8e92-c0044d151af3" (UID: "14b3a283-8e07-4350-8e92-c0044d151af3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:37:38.174186 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.174157    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14b3a283-8e07-4350-8e92-c0044d151af3-kube-api-access-xq47p" (OuterVolumeSpecName: "kube-api-access-xq47p") pod "14b3a283-8e07-4350-8e92-c0044d151af3" (UID: "14b3a283-8e07-4350-8e92-c0044d151af3"). InnerVolumeSpecName "kube-api-access-xq47p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:37:38.174644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.174620    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14b3a283-8e07-4350-8e92-c0044d151af3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "14b3a283-8e07-4350-8e92-c0044d151af3" (UID: "14b3a283-8e07-4350-8e92-c0044d151af3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:37:38.174758 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.174661    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14b3a283-8e07-4350-8e92-c0044d151af3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "14b3a283-8e07-4350-8e92-c0044d151af3" (UID: "14b3a283-8e07-4350-8e92-c0044d151af3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:37:38.272557 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.272523    2545 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:37:38.272557 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.272552    2545 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14b3a283-8e07-4350-8e92-c0044d151af3-console-serving-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:37:38.272557 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.272562    2545 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-oauth-serving-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:37:38.272780 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.272571    2545 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-console-config\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:37:38.272780 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.272580    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq47p\" (UniqueName: \"kubernetes.io/projected/14b3a283-8e07-4350-8e92-c0044d151af3-kube-api-access-xq47p\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:37:38.272780 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.272588    2545 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14b3a283-8e07-4350-8e92-c0044d151af3-console-oauth-config\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:37:38.272780 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.272596    2545 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14b3a283-8e07-4350-8e92-c0044d151af3-trusted-ca-bundle\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:37:38.864410 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.864381    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"]
Sep 11 16:37:38.864743 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.864731    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14b3a283-8e07-4350-8e92-c0044d151af3" containerName="console"
Sep 11 16:37:38.864794 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.864744    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="14b3a283-8e07-4350-8e92-c0044d151af3" containerName="console"
Sep 11 16:37:38.864828 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.864807    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="14b3a283-8e07-4350-8e92-c0044d151af3" containerName="console"
Sep 11 16:37:38.868004 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.867987    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"
Sep 11 16:37:38.871419 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.871399    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hzxn5\""
Sep 11 16:37:38.871649 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.871630    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 16:37:38.871739 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.871669    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 16:37:38.875793 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.875773    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/589ab627-9b5f-4456-901b-7233b0e911c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-s965r\" (UID: \"589ab627-9b5f-4456-901b-7233b0e911c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"
Sep 11 16:37:38.875893 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.875815    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t62x\" (UniqueName: \"kubernetes.io/projected/589ab627-9b5f-4456-901b-7233b0e911c6-kube-api-access-6t62x\") pod \"kuadrant-operator-controller-manager-5769f99d9f-s965r\" (UID: \"589ab627-9b5f-4456-901b-7233b0e911c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"
Sep 11 16:37:38.894013 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.893990    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"]
Sep 11 16:37:38.976654 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.976618    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t62x\" (UniqueName: \"kubernetes.io/projected/589ab627-9b5f-4456-901b-7233b0e911c6-kube-api-access-6t62x\") pod \"kuadrant-operator-controller-manager-5769f99d9f-s965r\" (UID: \"589ab627-9b5f-4456-901b-7233b0e911c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"
Sep 11 16:37:38.976887 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.976695    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/589ab627-9b5f-4456-901b-7233b0e911c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-s965r\" (UID: \"589ab627-9b5f-4456-901b-7233b0e911c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"
Sep 11 16:37:38.977074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.977057    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/589ab627-9b5f-4456-901b-7233b0e911c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-s965r\" (UID: \"589ab627-9b5f-4456-901b-7233b0e911c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"
Sep 11 16:37:38.988231 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:38.988212    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t62x\" (UniqueName: \"kubernetes.io/projected/589ab627-9b5f-4456-901b-7233b0e911c6-kube-api-access-6t62x\") pod \"kuadrant-operator-controller-manager-5769f99d9f-s965r\" (UID: \"589ab627-9b5f-4456-901b-7233b0e911c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"
Sep 11 16:37:39.073274 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:39.073231    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695c5b8ff5-hsvqr" event={"ID":"14b3a283-8e07-4350-8e92-c0044d151af3","Type":"ContainerDied","Data":"a3367a728b22d19a50438fa730b637132d6748e4797e49ef7a2c8db076d3fc25"}
Sep 11 16:37:39.073460 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:39.073305    2545 scope.go:117] "RemoveContainer" containerID="ff734e49460a2c095ddfe26ab8586e4481387988e09c0a71958739241851c18a"
Sep 11 16:37:39.073460 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:39.073250    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695c5b8ff5-hsvqr"
Sep 11 16:37:39.108185 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:39.108159    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-695c5b8ff5-hsvqr"]
Sep 11 16:37:39.116624 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:39.116572    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-695c5b8ff5-hsvqr"]
Sep 11 16:37:39.178845 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:39.178805    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"
Sep 11 16:37:39.465059 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:39.465035    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"]
Sep 11 16:37:39.468085 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:37:39.468057    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod589ab627_9b5f_4456_901b_7233b0e911c6.slice/crio-02ecbb73ea5c094221cff68d3ee47def6b6a23eb6f305af3df34bec0f76d6708 WatchSource:0}: Error finding container 02ecbb73ea5c094221cff68d3ee47def6b6a23eb6f305af3df34bec0f76d6708: Status 404 returned error can't find the container with id 02ecbb73ea5c094221cff68d3ee47def6b6a23eb6f305af3df34bec0f76d6708
Sep 11 16:37:39.937765 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:39.937733    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14b3a283-8e07-4350-8e92-c0044d151af3" path="/var/lib/kubelet/pods/14b3a283-8e07-4350-8e92-c0044d151af3/volumes"
Sep 11 16:37:40.079723 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:40.079690    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r" event={"ID":"589ab627-9b5f-4456-901b-7233b0e911c6","Type":"ContainerStarted","Data":"02ecbb73ea5c094221cff68d3ee47def6b6a23eb6f305af3df34bec0f76d6708"}
Sep 11 16:37:44.107431 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:44.107401    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r" event={"ID":"589ab627-9b5f-4456-901b-7233b0e911c6","Type":"ContainerStarted","Data":"bda93c206f8227f00d02ed26d14befed0b6105ea9d1652a46b2eb8d0c391fd25"}
Sep 11 16:37:44.107800 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:44.107470    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"
Sep 11 16:37:44.132145 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:44.132094    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r" podStartSLOduration=2.309339966 podStartE2EDuration="6.13207436s" totalImagesPullingTime="3.822734394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:37:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:37:44.130564727 +0000 UTC m=+482.683734655" watchObservedRunningTime="2026-09-11 16:37:44.13207436 +0000 UTC m=+482.685244289"
Sep 11 16:37:55.114812 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:37:55.114782    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"
Sep 11 16:38:16.646714 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:16.646679    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:38:16.651471 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:16.651455    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"
Sep 11 16:38:16.653628 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:16.653606    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qmcwz\""
Sep 11 16:38:16.653725 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:16.653707    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 16:38:16.659455 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:16.659435    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:38:16.696793 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:16.696766    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:38:16.787391 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:16.787355    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m8hf\" (UniqueName: \"kubernetes.io/projected/3df23723-01c6-4388-9708-8179b50c71fb-kube-api-access-6m8hf\") pod \"limitador-limitador-84994f7ffc-qgz2d\" (UID: \"3df23723-01c6-4388-9708-8179b50c71fb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"
Sep 11 16:38:16.787549 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:16.787397    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3df23723-01c6-4388-9708-8179b50c71fb-config-file\") pod \"limitador-limitador-84994f7ffc-qgz2d\" (UID: \"3df23723-01c6-4388-9708-8179b50c71fb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"
Sep 11 16:38:16.888556 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:16.888521    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m8hf\" (UniqueName: \"kubernetes.io/projected/3df23723-01c6-4388-9708-8179b50c71fb-kube-api-access-6m8hf\") pod \"limitador-limitador-84994f7ffc-qgz2d\" (UID: \"3df23723-01c6-4388-9708-8179b50c71fb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"
Sep 11 16:38:16.888556 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:16.888560    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3df23723-01c6-4388-9708-8179b50c71fb-config-file\") pod \"limitador-limitador-84994f7ffc-qgz2d\" (UID: \"3df23723-01c6-4388-9708-8179b50c71fb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"
Sep 11 16:38:16.889172 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:16.889153    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3df23723-01c6-4388-9708-8179b50c71fb-config-file\") pod \"limitador-limitador-84994f7ffc-qgz2d\" (UID: \"3df23723-01c6-4388-9708-8179b50c71fb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"
Sep 11 16:38:16.899838 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:16.899777    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m8hf\" (UniqueName: \"kubernetes.io/projected/3df23723-01c6-4388-9708-8179b50c71fb-kube-api-access-6m8hf\") pod \"limitador-limitador-84994f7ffc-qgz2d\" (UID: \"3df23723-01c6-4388-9708-8179b50c71fb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"
Sep 11 16:38:16.962882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:16.962832    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"
Sep 11 16:38:17.258572 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:17.258540    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:38:17.262613 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:38:17.262586    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df23723_01c6_4388_9708_8179b50c71fb.slice/crio-f74e7dd73377403d0697a5cb24b917bb9ea24a5583592454bf76f8de9676d0dc WatchSource:0}: Error finding container f74e7dd73377403d0697a5cb24b917bb9ea24a5583592454bf76f8de9676d0dc: Status 404 returned error can't find the container with id f74e7dd73377403d0697a5cb24b917bb9ea24a5583592454bf76f8de9676d0dc
Sep 11 16:38:17.269442 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:17.269412    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-qgz2d" event={"ID":"3df23723-01c6-4388-9708-8179b50c71fb","Type":"ContainerStarted","Data":"f74e7dd73377403d0697a5cb24b917bb9ea24a5583592454bf76f8de9676d0dc"}
Sep 11 16:38:19.282589 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:19.282557    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-qgz2d" event={"ID":"3df23723-01c6-4388-9708-8179b50c71fb","Type":"ContainerStarted","Data":"e83a60604673ec9b12684b275e9d7b7cbfe8d772cfeb5a11058ae302e432feb4"}
Sep 11 16:38:19.282999 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:19.282615    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"
Sep 11 16:38:19.311234 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:19.311186    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-qgz2d" podStartSLOduration=2.263000559 podStartE2EDuration="3.311170312s" totalImagesPullingTime="1.048169753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:38:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:38:19.309053122 +0000 UTC m=+517.862223062" watchObservedRunningTime="2026-09-11 16:38:19.311170312 +0000 UTC m=+517.864340239"
Sep 11 16:38:30.288519 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:38:30.288492    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"
Sep 11 16:39:29.797087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:39:29.797055    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:39:41.902722 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:39:41.902690    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:39:41.903627 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:39:41.903597    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:39:49.871594 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:39:49.871558    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:40:10.836356 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.836262    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"]
Sep 11 16:40:10.839744 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.839727    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.846330 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.846065    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-47mg7\""
Sep 11 16:40:10.846330 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.846097    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 11 16:40:10.846765 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.846744    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 16:40:10.847066 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.847049    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 16:40:10.857974 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.857954    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"]
Sep 11 16:40:10.890351 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.890311    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ff49a99-a74b-4a40-95de-e3e53b32df77-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.890351 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.890347    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ff49a99-a74b-4a40-95de-e3e53b32df77-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.890575 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.890382    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzkt8\" (UniqueName: \"kubernetes.io/projected/9ff49a99-a74b-4a40-95de-e3e53b32df77-kube-api-access-wzkt8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.890575 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.890406    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ff49a99-a74b-4a40-95de-e3e53b32df77-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.890575 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.890519    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ff49a99-a74b-4a40-95de-e3e53b32df77-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.890575 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.890556    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ff49a99-a74b-4a40-95de-e3e53b32df77-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.991901 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.991848    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ff49a99-a74b-4a40-95de-e3e53b32df77-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.992060 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.992044    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ff49a99-a74b-4a40-95de-e3e53b32df77-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.992157 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.992074    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ff49a99-a74b-4a40-95de-e3e53b32df77-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.992894 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.992376    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzkt8\" (UniqueName: \"kubernetes.io/projected/9ff49a99-a74b-4a40-95de-e3e53b32df77-kube-api-access-wzkt8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.992894 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.992440    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ff49a99-a74b-4a40-95de-e3e53b32df77-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.992894 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.992587    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ff49a99-a74b-4a40-95de-e3e53b32df77-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.992894 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.992731    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ff49a99-a74b-4a40-95de-e3e53b32df77-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.992894 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.992656    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ff49a99-a74b-4a40-95de-e3e53b32df77-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.992894 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.992648    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ff49a99-a74b-4a40-95de-e3e53b32df77-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.994353 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.994335    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ff49a99-a74b-4a40-95de-e3e53b32df77-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:10.995209 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:10.995186    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ff49a99-a74b-4a40-95de-e3e53b32df77-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:11.002826 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:11.002808    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzkt8\" (UniqueName: \"kubernetes.io/projected/9ff49a99-a74b-4a40-95de-e3e53b32df77-kube-api-access-wzkt8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k7mcb\" (UID: \"9ff49a99-a74b-4a40-95de-e3e53b32df77\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:11.149576 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:11.149504    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:11.428183 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:11.428156    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"]
Sep 11 16:40:11.430297 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:40:11.430270    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ff49a99_a74b_4a40_95de_e3e53b32df77.slice/crio-8ac8ce51be485f5d8952565abb2e7b77fc13089218ddca64c688ab6638c2f2a7 WatchSource:0}: Error finding container 8ac8ce51be485f5d8952565abb2e7b77fc13089218ddca64c688ab6638c2f2a7: Status 404 returned error can't find the container with id 8ac8ce51be485f5d8952565abb2e7b77fc13089218ddca64c688ab6638c2f2a7
Sep 11 16:40:11.431584 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:11.431563    2545 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 16:40:11.455432 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:11.455404    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:40:11.847973 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:11.847932    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb" event={"ID":"9ff49a99-a74b-4a40-95de-e3e53b32df77","Type":"ContainerStarted","Data":"8ac8ce51be485f5d8952565abb2e7b77fc13089218ddca64c688ab6638c2f2a7"}
Sep 11 16:40:12.856500 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:12.856468    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb" event={"ID":"9ff49a99-a74b-4a40-95de-e3e53b32df77","Type":"ContainerStarted","Data":"651fda4a6911064bb8115d8ddd6f1fd25f979633ea009a605cca6146383b8aca"}
Sep 11 16:40:12.879740 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:12.879693    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb" podStartSLOduration=1.676678161 podStartE2EDuration="2.879677833s" totalImagesPullingTime="1.202999672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:40:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:40:12.878100801 +0000 UTC m=+631.431270746" watchObservedRunningTime="2026-09-11 16:40:12.879677833 +0000 UTC m=+631.432847760"
Sep 11 16:40:21.149670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:21.149621    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:21.150119 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:21.149727    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:21.162327 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:21.162305    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:21.911845 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:21.911814    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k7mcb"
Sep 11 16:40:23.245902 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:23.245873    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:40:31.144436 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.144399    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"]
Sep 11 16:40:31.148113 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.148097    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.156142 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.156116    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 11 16:40:31.174851 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.174827    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"]
Sep 11 16:40:31.263362 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.263334    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7aba878-c7c8-481f-bc5c-074d731aad3d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.263527 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.263396    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7aba878-c7c8-481f-bc5c-074d731aad3d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.263527 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.263422    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69z8s\" (UniqueName: \"kubernetes.io/projected/c7aba878-c7c8-481f-bc5c-074d731aad3d-kube-api-access-69z8s\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.263527 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.263448    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7aba878-c7c8-481f-bc5c-074d731aad3d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.263527 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.263468    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7aba878-c7c8-481f-bc5c-074d731aad3d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.263527 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.263485    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7aba878-c7c8-481f-bc5c-074d731aad3d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.364391 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.364363    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7aba878-c7c8-481f-bc5c-074d731aad3d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.364554 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.364400    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69z8s\" (UniqueName: \"kubernetes.io/projected/c7aba878-c7c8-481f-bc5c-074d731aad3d-kube-api-access-69z8s\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.364554 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.364418    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7aba878-c7c8-481f-bc5c-074d731aad3d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.364554 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.364518    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7aba878-c7c8-481f-bc5c-074d731aad3d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.364554 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.364545    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7aba878-c7c8-481f-bc5c-074d731aad3d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.364772 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.364597    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7aba878-c7c8-481f-bc5c-074d731aad3d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.364985 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.364957    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7aba878-c7c8-481f-bc5c-074d731aad3d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.365074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.364998    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7aba878-c7c8-481f-bc5c-074d731aad3d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.365074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.365001    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7aba878-c7c8-481f-bc5c-074d731aad3d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.366926 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.366897    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7aba878-c7c8-481f-bc5c-074d731aad3d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.367174 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.367155    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7aba878-c7c8-481f-bc5c-074d731aad3d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.374491 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.374469    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69z8s\" (UniqueName: \"kubernetes.io/projected/c7aba878-c7c8-481f-bc5c-074d731aad3d-kube-api-access-69z8s\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wb4md\" (UID: \"c7aba878-c7c8-481f-bc5c-074d731aad3d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.458058 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.457981    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:31.734095 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.734072    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"]
Sep 11 16:40:31.736327 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:40:31.736299    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7aba878_c7c8_481f_bc5c_074d731aad3d.slice/crio-fab3a7cb125d50229ea065dde6477af59d36493af177927cea5d5470b51e2957 WatchSource:0}: Error finding container fab3a7cb125d50229ea065dde6477af59d36493af177927cea5d5470b51e2957: Status 404 returned error can't find the container with id fab3a7cb125d50229ea065dde6477af59d36493af177927cea5d5470b51e2957
Sep 11 16:40:31.951770 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:31.951735    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md" event={"ID":"c7aba878-c7c8-481f-bc5c-074d731aad3d","Type":"ContainerStarted","Data":"fab3a7cb125d50229ea065dde6477af59d36493af177927cea5d5470b51e2957"}
Sep 11 16:40:32.462106 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:32.462062    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:40:32.958958 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:32.958919    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md" event={"ID":"c7aba878-c7c8-481f-bc5c-074d731aad3d","Type":"ContainerStarted","Data":"0e6ec5b53edb160cf3482c594efe8f484257b22d8d0b062d369adf8e8b15d03a"}
Sep 11 16:40:32.980139 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:32.980097    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md" podStartSLOduration=1.785438278 podStartE2EDuration="1.980082562s" totalImagesPullingTime="194.644284ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:40:32.979959228 +0000 UTC m=+651.533129172" watchObservedRunningTime="2026-09-11 16:40:32.980082562 +0000 UTC m=+651.533252486"
Sep 11 16:40:38.746461 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:38.746419    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:40:41.458650 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:41.458613    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:41.459045 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:41.458699    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:41.471091 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:41.471068    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:42.014325 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:42.014300    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wb4md"
Sep 11 16:40:42.974490 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:42.974455    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:40:52.410023 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.409985    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"]
Sep 11 16:40:52.415091 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.415071    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.417369 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.417347    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 11 16:40:52.422018 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.422000    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"]
Sep 11 16:40:52.541783 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.541740    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25680cbe-a8cb-4083-b794-ebcd1fe40e83-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.541783 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.541784    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25680cbe-a8cb-4083-b794-ebcd1fe40e83-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.542070 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.541838    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25680cbe-a8cb-4083-b794-ebcd1fe40e83-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.542070 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.541980    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb48f\" (UniqueName: \"kubernetes.io/projected/25680cbe-a8cb-4083-b794-ebcd1fe40e83-kube-api-access-mb48f\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.542070 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.542038    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25680cbe-a8cb-4083-b794-ebcd1fe40e83-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.542194 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.542069    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25680cbe-a8cb-4083-b794-ebcd1fe40e83-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.642710 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.642666    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25680cbe-a8cb-4083-b794-ebcd1fe40e83-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.642933 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.642754    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb48f\" (UniqueName: \"kubernetes.io/projected/25680cbe-a8cb-4083-b794-ebcd1fe40e83-kube-api-access-mb48f\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.642933 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.642780    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25680cbe-a8cb-4083-b794-ebcd1fe40e83-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.642933 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.642797    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25680cbe-a8cb-4083-b794-ebcd1fe40e83-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.642933 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.642816    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25680cbe-a8cb-4083-b794-ebcd1fe40e83-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.642933 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.642836    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25680cbe-a8cb-4083-b794-ebcd1fe40e83-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.643245 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.643169    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25680cbe-a8cb-4083-b794-ebcd1fe40e83-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.643282 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.643239    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25680cbe-a8cb-4083-b794-ebcd1fe40e83-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.643315 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.643296    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25680cbe-a8cb-4083-b794-ebcd1fe40e83-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.645415 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.645390    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25680cbe-a8cb-4083-b794-ebcd1fe40e83-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.645807 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.645781    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25680cbe-a8cb-4083-b794-ebcd1fe40e83-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.652193 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.652177    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb48f\" (UniqueName: \"kubernetes.io/projected/25680cbe-a8cb-4083-b794-ebcd1fe40e83-kube-api-access-mb48f\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6\" (UID: \"25680cbe-a8cb-4083-b794-ebcd1fe40e83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:52.725308 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:52.725232    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:40:53.003887 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:53.003840    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"]
Sep 11 16:40:53.005932 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:40:53.005830    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25680cbe_a8cb_4083_b794_ebcd1fe40e83.slice/crio-bb620dea9acc64a5dca87adb919df54081f6be5157b7553da637ae9587efbb5a WatchSource:0}: Error finding container bb620dea9acc64a5dca87adb919df54081f6be5157b7553da637ae9587efbb5a: Status 404 returned error can't find the container with id bb620dea9acc64a5dca87adb919df54081f6be5157b7553da637ae9587efbb5a
Sep 11 16:40:53.058483 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:53.058447    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6" event={"ID":"25680cbe-a8cb-4083-b794-ebcd1fe40e83","Type":"ContainerStarted","Data":"bb620dea9acc64a5dca87adb919df54081f6be5157b7553da637ae9587efbb5a"}
Sep 11 16:40:54.064825 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:54.064794    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6" event={"ID":"25680cbe-a8cb-4083-b794-ebcd1fe40e83","Type":"ContainerStarted","Data":"d3cfd67b18134149793d52b5126df47c46158e38d3f669b2f776d0ade10c4e40"}
Sep 11 16:40:54.086484 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:54.086431    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6" podStartSLOduration=1.89068959 podStartE2EDuration="2.086412202s" totalImagesPullingTime="195.722612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:40:54.084840297 +0000 UTC m=+672.638010240" watchObservedRunningTime="2026-09-11 16:40:54.086412202 +0000 UTC m=+672.639582128"
Sep 11 16:40:54.743542 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:40:54.743509    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:41:02.725511 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:41:02.725475    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:41:02.726032 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:41:02.725524    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:41:02.737883 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:41:02.737841    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:41:03.120973 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:41:03.120947    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6"
Sep 11 16:42:40.244938 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:40.244908    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7792-70357f"]
Sep 11 16:42:40.248142 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:40.248126    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7792-70357f"
Sep 11 16:42:40.250368 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:40.250346    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vtvd2\""
Sep 11 16:42:40.257944 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:40.257921    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7792-70357f"]
Sep 11 16:42:40.335150 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:40.335126    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr5nl\" (UniqueName: \"kubernetes.io/projected/f8940edc-174b-47ef-a73e-a24bf46289fe-kube-api-access-xr5nl\") pod \"test-curl-7792-70357f\" (UID: \"f8940edc-174b-47ef-a73e-a24bf46289fe\") " pod="opendatahub/test-curl-7792-70357f"
Sep 11 16:42:40.435677 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:40.435652    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr5nl\" (UniqueName: \"kubernetes.io/projected/f8940edc-174b-47ef-a73e-a24bf46289fe-kube-api-access-xr5nl\") pod \"test-curl-7792-70357f\" (UID: \"f8940edc-174b-47ef-a73e-a24bf46289fe\") " pod="opendatahub/test-curl-7792-70357f"
Sep 11 16:42:40.445171 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:40.445144    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr5nl\" (UniqueName: \"kubernetes.io/projected/f8940edc-174b-47ef-a73e-a24bf46289fe-kube-api-access-xr5nl\") pod \"test-curl-7792-70357f\" (UID: \"f8940edc-174b-47ef-a73e-a24bf46289fe\") " pod="opendatahub/test-curl-7792-70357f"
Sep 11 16:42:40.557978 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:40.557927    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7792-70357f"
Sep 11 16:42:40.828233 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:40.828169    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7792-70357f"]
Sep 11 16:42:40.830719 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:42:40.830693    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8940edc_174b_47ef_a73e_a24bf46289fe.slice/crio-1d6038cff3aab9aa5e28e479b8a2165760137f736f06c1df46aec2223ef3501e WatchSource:0}: Error finding container 1d6038cff3aab9aa5e28e479b8a2165760137f736f06c1df46aec2223ef3501e: Status 404 returned error can't find the container with id 1d6038cff3aab9aa5e28e479b8a2165760137f736f06c1df46aec2223ef3501e
Sep 11 16:42:41.616345 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:41.616306    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7792-70357f" event={"ID":"f8940edc-174b-47ef-a73e-a24bf46289fe","Type":"ContainerStarted","Data":"1d6038cff3aab9aa5e28e479b8a2165760137f736f06c1df46aec2223ef3501e"}
Sep 11 16:42:41.850494 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:41.850457    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:42:42.622922 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:42.622889    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7792-70357f" event={"ID":"f8940edc-174b-47ef-a73e-a24bf46289fe","Type":"ContainerStarted","Data":"62d617404430c428680c1dc2b50fb8bb993a1cdc3d6d5b43196b4a6dc3a57824"}
Sep 11 16:42:43.271286 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:43.271221    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7792-70357f" podStartSLOduration=2.451975363 podStartE2EDuration="3.271199664s" totalImagesPullingTime="819.224301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:42:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:42:42.647073666 +0000 UTC m=+781.200243593" watchObservedRunningTime="2026-09-11 16:42:43.271199664 +0000 UTC m=+781.824369592"
Sep 11 16:42:43.271743 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:43.271725    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7792-70357f"]
Sep 11 16:42:43.277011 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:43.276979    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7792-70357f"]
Sep 11 16:42:43.399130 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:43.399060    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7792-244f02"]
Sep 11 16:42:43.399504 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:43.399489    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8940edc-174b-47ef-a73e-a24bf46289fe" containerName="test-curl-7792-70357f"
Sep 11 16:42:43.399545 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:43.399508    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8940edc-174b-47ef-a73e-a24bf46289fe" containerName="test-curl-7792-70357f"
Sep 11 16:42:43.399601 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:43.399592    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8940edc-174b-47ef-a73e-a24bf46289fe" containerName="test-curl-7792-70357f"
Sep 11 16:42:43.402911 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:43.402894    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7792-244f02"
Sep 11 16:42:43.409989 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:43.409966    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7792-244f02"]
Sep 11 16:42:43.461598 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:43.461567    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhcxg\" (UniqueName: \"kubernetes.io/projected/c9b3f182-dce6-4113-a2b9-dfafc531b087-kube-api-access-hhcxg\") pod \"test-curl-7792-244f02\" (UID: \"c9b3f182-dce6-4113-a2b9-dfafc531b087\") " pod="opendatahub/test-curl-7792-244f02"
Sep 11 16:42:43.562108 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:43.562076    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhcxg\" (UniqueName: \"kubernetes.io/projected/c9b3f182-dce6-4113-a2b9-dfafc531b087-kube-api-access-hhcxg\") pod \"test-curl-7792-244f02\" (UID: \"c9b3f182-dce6-4113-a2b9-dfafc531b087\") " pod="opendatahub/test-curl-7792-244f02"
Sep 11 16:42:43.574019 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:43.573993    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhcxg\" (UniqueName: \"kubernetes.io/projected/c9b3f182-dce6-4113-a2b9-dfafc531b087-kube-api-access-hhcxg\") pod \"test-curl-7792-244f02\" (UID: \"c9b3f182-dce6-4113-a2b9-dfafc531b087\") " pod="opendatahub/test-curl-7792-244f02"
Sep 11 16:42:43.628323 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:42:43.628298    2545 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7792-70357f" container="test-curl-7792-70357f"
Sep 11 16:42:43.714272 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:43.714188    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7792-244f02"
Sep 11 16:42:43.989845 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:43.989824    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7792-244f02"]
Sep 11 16:42:43.992559 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:42:43.992529    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9b3f182_dce6_4113_a2b9_dfafc531b087.slice/crio-caa201ef5fdb0dd2ea3d0be1fdb9b3366a7bb78adae04fcb16666e92e65f5ba6 WatchSource:0}: Error finding container caa201ef5fdb0dd2ea3d0be1fdb9b3366a7bb78adae04fcb16666e92e65f5ba6: Status 404 returned error can't find the container with id caa201ef5fdb0dd2ea3d0be1fdb9b3366a7bb78adae04fcb16666e92e65f5ba6
Sep 11 16:42:44.634641 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:44.634607    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7792-244f02" event={"ID":"c9b3f182-dce6-4113-a2b9-dfafc531b087","Type":"ContainerStarted","Data":"3fb00d69c44532e39ba1fe4520186b89c33050f75c0fed0c4b870aa703210ea7"}
Sep 11 16:42:44.634641 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:44.634649    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7792-244f02" event={"ID":"c9b3f182-dce6-4113-a2b9-dfafc531b087","Type":"ContainerStarted","Data":"caa201ef5fdb0dd2ea3d0be1fdb9b3366a7bb78adae04fcb16666e92e65f5ba6"}
Sep 11 16:42:44.635160 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:44.634719    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7792-70357f" podUID="f8940edc-174b-47ef-a73e-a24bf46289fe" containerName="test-curl-7792-70357f" containerID="cri-o://62d617404430c428680c1dc2b50fb8bb993a1cdc3d6d5b43196b4a6dc3a57824" gracePeriod=2
Sep 11 16:42:44.651396 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:44.651356    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7792-244f02" podStartSLOduration=1.6513410130000001 podStartE2EDuration="1.651341013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:42:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:42:44.649411294 +0000 UTC m=+783.202581221" watchObservedRunningTime="2026-09-11 16:42:44.651341013 +0000 UTC m=+783.204510939"
Sep 11 16:42:45.241996 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:45.241962    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7792-244f02"]
Sep 11 16:42:45.249069 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:45.249039    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7792-244f02"]
Sep 11 16:42:45.640254 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:42:45.640221    2545 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7792-244f02" container="test-curl-7792-244f02"
Sep 11 16:42:46.061306 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.061275    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"]
Sep 11 16:42:46.061698 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.061684    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9b3f182-dce6-4113-a2b9-dfafc531b087" containerName="test-curl-7792-244f02"
Sep 11 16:42:46.061749 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.061699    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9b3f182-dce6-4113-a2b9-dfafc531b087" containerName="test-curl-7792-244f02"
Sep 11 16:42:46.061803 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.061759    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9b3f182-dce6-4113-a2b9-dfafc531b087" containerName="test-curl-7792-244f02"
Sep 11 16:42:46.064690 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.064673    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.066814 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.066792    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-42752c-openshift-default-dockercfg-djj2p\""
Sep 11 16:42:46.085943 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.085646    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"]
Sep 11 16:42:46.187108 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.187073    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-token\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.187285 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.187118    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph874\" (UniqueName: \"kubernetes.io/projected/6d250f02-6408-42a9-8183-7b7bbdd723f1-kube-api-access-ph874\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.187285 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.187158    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-credential-socket\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.187285 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.187227    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-podinfo\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.187285 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.187278    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-workload-certs\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.187444 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.187316    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-data\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.187444 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.187344    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d250f02-6408-42a9-8183-7b7bbdd723f1-istiod-ca-cert\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.187444 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.187370    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-workload-socket\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.187444 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.187398    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-envoy\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.288264 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.288233    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph874\" (UniqueName: \"kubernetes.io/projected/6d250f02-6408-42a9-8183-7b7bbdd723f1-kube-api-access-ph874\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.288423 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.288283    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-credential-socket\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.288423 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.288316    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-podinfo\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.288423 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.288340    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-workload-certs\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.288423 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.288367    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-data\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.288423 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.288393    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d250f02-6408-42a9-8183-7b7bbdd723f1-istiod-ca-cert\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.288658 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.288510    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-workload-socket\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.288658 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.288559    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-envoy\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.288658 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.288651    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-token\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.288901 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.288830    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-workload-certs\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.289003 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.288904    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-credential-socket\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.289059 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.288996    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-data\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.289059 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.289033    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-workload-socket\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.289205 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.289185    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d250f02-6408-42a9-8183-7b7bbdd723f1-istiod-ca-cert\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.291080 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.291059    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-envoy\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.291176 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.291146    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-podinfo\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.302123 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.302097    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-token\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.303837 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.303807    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph874\" (UniqueName: \"kubernetes.io/projected/6d250f02-6408-42a9-8183-7b7bbdd723f1-kube-api-access-ph874\") pod \"e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") " pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.375412 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.375328    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:46.646492 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.646438    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7792-244f02" podUID="c9b3f182-dce6-4113-a2b9-dfafc531b087" containerName="test-curl-7792-244f02" containerID="cri-o://3fb00d69c44532e39ba1fe4520186b89c33050f75c0fed0c4b870aa703210ea7" gracePeriod=2
Sep 11 16:42:46.655968 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.655950    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"]
Sep 11 16:42:46.666911 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:42:46.666859    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d250f02_6408_42a9_8183_7b7bbdd723f1.slice/crio-3d2eb2a858ffe4a299c8abd24cbb6803997c0b6b3b907cedcb2f9740b5283f71 WatchSource:0}: Error finding container 3d2eb2a858ffe4a299c8abd24cbb6803997c0b6b3b907cedcb2f9740b5283f71: Status 404 returned error can't find the container with id 3d2eb2a858ffe4a299c8abd24cbb6803997c0b6b3b907cedcb2f9740b5283f71
Sep 11 16:42:46.669269 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.669234    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:42:46.669346 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.669329    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:42:46.669390 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.669374    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:42:46.786685 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.786659    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7792-70357f"
Sep 11 16:42:46.894382 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.894295    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xr5nl\" (UniqueName: \"kubernetes.io/projected/f8940edc-174b-47ef-a73e-a24bf46289fe-kube-api-access-xr5nl\") pod \"f8940edc-174b-47ef-a73e-a24bf46289fe\" (UID: \"f8940edc-174b-47ef-a73e-a24bf46289fe\") "
Sep 11 16:42:46.896498 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.896473    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8940edc-174b-47ef-a73e-a24bf46289fe-kube-api-access-xr5nl" (OuterVolumeSpecName: "kube-api-access-xr5nl") pod "f8940edc-174b-47ef-a73e-a24bf46289fe" (UID: "f8940edc-174b-47ef-a73e-a24bf46289fe"). InnerVolumeSpecName "kube-api-access-xr5nl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:42:46.996074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:46.996041    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr5nl\" (UniqueName: \"kubernetes.io/projected/f8940edc-174b-47ef-a73e-a24bf46289fe-kube-api-access-xr5nl\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:42:47.228216 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.228135    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"]
Sep 11 16:42:47.231814 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.231796    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.236610 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.236592    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-845e25d3-openshift-default-dockercfg-zg2q6\""
Sep 11 16:42:47.249952 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.249928    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"]
Sep 11 16:42:47.299098 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.299069    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-workload-socket\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.299198 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.299132    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5d2l\" (UniqueName: \"kubernetes.io/projected/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-kube-api-access-k5d2l\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.299198 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.299173    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-credential-socket\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.299314 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.299231    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-envoy\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.299314 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.299256    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istiod-ca-cert\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.299403 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.299387    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-workload-certs\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.299456 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.299415    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-data\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.299548 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.299527    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-podinfo\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.299595 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.299581    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-token\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.400291 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.400248    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-envoy\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.400291 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.400286    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istiod-ca-cert\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.400503 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.400322    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-workload-certs\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.400503 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.400341    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-data\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.400503 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.400427    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-podinfo\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.400503 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.400465    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-token\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.400694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.400541    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-workload-socket\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.400694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.400574    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5d2l\" (UniqueName: \"kubernetes.io/projected/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-kube-api-access-k5d2l\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.400790 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.400736    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-credential-socket\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.400839 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.400802    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-data\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.400839 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.400742    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-workload-certs\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.401151 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.401104    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-credential-socket\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.401151 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.401127    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istiod-ca-cert\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.401289 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.401142    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-workload-socket\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.402712 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.402689    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-envoy\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.402953 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.402933    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-podinfo\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.409704 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.409681    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5d2l\" (UniqueName: \"kubernetes.io/projected/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-kube-api-access-k5d2l\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.409843 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.409731    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-token\") pod \"e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") " pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.541469 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.541442    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:47.653625 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.653592    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5" event={"ID":"6d250f02-6408-42a9-8183-7b7bbdd723f1","Type":"ContainerStarted","Data":"f4780dde6baf8b5dec3b007986d2537f12b64fc2dbc53718a13ecbb3b0261e72"}
Sep 11 16:42:47.654075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.653646    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5" event={"ID":"6d250f02-6408-42a9-8183-7b7bbdd723f1","Type":"ContainerStarted","Data":"3d2eb2a858ffe4a299c8abd24cbb6803997c0b6b3b907cedcb2f9740b5283f71"}
Sep 11 16:42:47.654809 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.654785    2545 generic.go:356] "Generic (PLEG): container finished" podID="f8940edc-174b-47ef-a73e-a24bf46289fe" containerID="62d617404430c428680c1dc2b50fb8bb993a1cdc3d6d5b43196b4a6dc3a57824" exitCode=137
Sep 11 16:42:47.654916 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.654829    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7792-70357f"
Sep 11 16:42:47.654916 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.654894    2545 scope.go:117] "RemoveContainer" containerID="62d617404430c428680c1dc2b50fb8bb993a1cdc3d6d5b43196b4a6dc3a57824"
Sep 11 16:42:47.664347 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.664327    2545 scope.go:117] "RemoveContainer" containerID="62d617404430c428680c1dc2b50fb8bb993a1cdc3d6d5b43196b4a6dc3a57824"
Sep 11 16:42:47.664601 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:42:47.664584    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62d617404430c428680c1dc2b50fb8bb993a1cdc3d6d5b43196b4a6dc3a57824\": container with ID starting with 62d617404430c428680c1dc2b50fb8bb993a1cdc3d6d5b43196b4a6dc3a57824 not found: ID does not exist" containerID="62d617404430c428680c1dc2b50fb8bb993a1cdc3d6d5b43196b4a6dc3a57824"
Sep 11 16:42:47.664648 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.664611    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d617404430c428680c1dc2b50fb8bb993a1cdc3d6d5b43196b4a6dc3a57824"} err="failed to get container status \"62d617404430c428680c1dc2b50fb8bb993a1cdc3d6d5b43196b4a6dc3a57824\": rpc error: code = NotFound desc = could not find container \"62d617404430c428680c1dc2b50fb8bb993a1cdc3d6d5b43196b4a6dc3a57824\": container with ID starting with 62d617404430c428680c1dc2b50fb8bb993a1cdc3d6d5b43196b4a6dc3a57824 not found: ID does not exist"
Sep 11 16:42:47.676186 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.676139    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5" podStartSLOduration=1.676120643 podStartE2EDuration="1.676120643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:42:47.674391684 +0000 UTC m=+786.227561610" watchObservedRunningTime="2026-09-11 16:42:47.676120643 +0000 UTC m=+786.229290570"
Sep 11 16:42:47.678326 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:42:47.678292    2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7792-70357f\" is forbidden: User \"system:node:ip-10-0-135-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-58.ec2.internal' and this object" podUID="f8940edc-174b-47ef-a73e-a24bf46289fe" pod="opendatahub/test-curl-7792-70357f"
Sep 11 16:42:47.824858 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.824783    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"]
Sep 11 16:42:47.827974 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:42:47.827941    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod690ceb1a_1b49_48eb_9d51_c18a377cbb4b.slice/crio-ad62ff3812ea53def5d4d758601f89edaac26c2fa2aa7ec1147ed23aedcf975e WatchSource:0}: Error finding container ad62ff3812ea53def5d4d758601f89edaac26c2fa2aa7ec1147ed23aedcf975e: Status 404 returned error can't find the container with id ad62ff3812ea53def5d4d758601f89edaac26c2fa2aa7ec1147ed23aedcf975e
Sep 11 16:42:47.832839 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.832809    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:42:47.832931 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.832915    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:42:47.832979 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.832956    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:42:47.940551 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:47.940516    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8940edc-174b-47ef-a73e-a24bf46289fe" path="/var/lib/kubelet/pods/f8940edc-174b-47ef-a73e-a24bf46289fe/volumes"
Sep 11 16:42:48.376011 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:48.375981    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:48.381079 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:48.381053    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:48.663889 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:48.663841    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk" event={"ID":"690ceb1a-1b49-48eb-9d51-c18a377cbb4b","Type":"ContainerStarted","Data":"955443170f1b93c43b4d7185eded13252c40767f7386280dd06b84b2580bcbfc"}
Sep 11 16:42:48.664222 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:48.663906    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk" event={"ID":"690ceb1a-1b49-48eb-9d51-c18a377cbb4b","Type":"ContainerStarted","Data":"ad62ff3812ea53def5d4d758601f89edaac26c2fa2aa7ec1147ed23aedcf975e"}
Sep 11 16:42:48.664222 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:48.664105    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:48.665061 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:48.665045    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:42:48.686001 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:48.685957    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk" podStartSLOduration=1.685940566 podStartE2EDuration="1.685940566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:42:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:42:48.685186018 +0000 UTC m=+787.238355975" watchObservedRunningTime="2026-09-11 16:42:48.685940566 +0000 UTC m=+787.239110517"
Sep 11 16:42:48.827538 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:48.827512    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7792-244f02"
Sep 11 16:42:48.914628 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:48.914556    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhcxg\" (UniqueName: \"kubernetes.io/projected/c9b3f182-dce6-4113-a2b9-dfafc531b087-kube-api-access-hhcxg\") pod \"c9b3f182-dce6-4113-a2b9-dfafc531b087\" (UID: \"c9b3f182-dce6-4113-a2b9-dfafc531b087\") "
Sep 11 16:42:48.916740 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:48.916709    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9b3f182-dce6-4113-a2b9-dfafc531b087-kube-api-access-hhcxg" (OuterVolumeSpecName: "kube-api-access-hhcxg") pod "c9b3f182-dce6-4113-a2b9-dfafc531b087" (UID: "c9b3f182-dce6-4113-a2b9-dfafc531b087"). InnerVolumeSpecName "kube-api-access-hhcxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:42:49.016236 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:49.016210    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhcxg\" (UniqueName: \"kubernetes.io/projected/c9b3f182-dce6-4113-a2b9-dfafc531b087-kube-api-access-hhcxg\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:42:49.542113 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:49.542078    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:49.547656 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:49.547629    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:49.671264 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:49.671231    2545 generic.go:356] "Generic (PLEG): container finished" podID="c9b3f182-dce6-4113-a2b9-dfafc531b087" containerID="3fb00d69c44532e39ba1fe4520186b89c33050f75c0fed0c4b870aa703210ea7" exitCode=137
Sep 11 16:42:49.671662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:49.671269    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7792-244f02"
Sep 11 16:42:49.671662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:49.671326    2545 scope.go:117] "RemoveContainer" containerID="3fb00d69c44532e39ba1fe4520186b89c33050f75c0fed0c4b870aa703210ea7"
Sep 11 16:42:49.671936 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:49.671917    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:49.672762 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:49.672672    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:42:49.680334 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:49.680318    2545 scope.go:117] "RemoveContainer" containerID="3fb00d69c44532e39ba1fe4520186b89c33050f75c0fed0c4b870aa703210ea7"
Sep 11 16:42:49.680579 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:42:49.680561    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fb00d69c44532e39ba1fe4520186b89c33050f75c0fed0c4b870aa703210ea7\": container with ID starting with 3fb00d69c44532e39ba1fe4520186b89c33050f75c0fed0c4b870aa703210ea7 not found: ID does not exist" containerID="3fb00d69c44532e39ba1fe4520186b89c33050f75c0fed0c4b870aa703210ea7"
Sep 11 16:42:49.680654 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:49.680585    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fb00d69c44532e39ba1fe4520186b89c33050f75c0fed0c4b870aa703210ea7"} err="failed to get container status \"3fb00d69c44532e39ba1fe4520186b89c33050f75c0fed0c4b870aa703210ea7\": rpc error: code = NotFound desc = could not find container \"3fb00d69c44532e39ba1fe4520186b89c33050f75c0fed0c4b870aa703210ea7\": container with ID starting with 3fb00d69c44532e39ba1fe4520186b89c33050f75c0fed0c4b870aa703210ea7 not found: ID does not exist"
Sep 11 16:42:49.748987 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:42:49.748959    2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7792-244f02\" is forbidden: User \"system:node:ip-10-0-135-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-58.ec2.internal' and this object" podUID="c9b3f182-dce6-4113-a2b9-dfafc531b087" pod="opendatahub/test-curl-7792-244f02"
Sep 11 16:42:49.937687 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:49.937615    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9b3f182-dce6-4113-a2b9-dfafc531b087" path="/var/lib/kubelet/pods/c9b3f182-dce6-4113-a2b9-dfafc531b087/volumes"
Sep 11 16:42:51.702729 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:51.702698    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"]
Sep 11 16:42:51.707486 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:51.707471    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"
Sep 11 16:42:51.710020 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:51.709998    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-42752c-dockercfg-pwnfh\""
Sep 11 16:42:51.710119 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:51.710104    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-42752c\""
Sep 11 16:42:51.717124 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:51.717100    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"]
Sep 11 16:42:51.741703 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:51.741668    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w88s\" (UniqueName: \"kubernetes.io/projected/6530b7d2-6dc7-4792-a774-61673f74d10e-kube-api-access-7w88s\") pod \"payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs\" (UID: \"6530b7d2-6dc7-4792-a774-61673f74d10e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"
Sep 11 16:42:51.741835 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:51.741732    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6530b7d2-6dc7-4792-a774-61673f74d10e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs\" (UID: \"6530b7d2-6dc7-4792-a774-61673f74d10e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"
Sep 11 16:42:51.842522 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:51.842487    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w88s\" (UniqueName: \"kubernetes.io/projected/6530b7d2-6dc7-4792-a774-61673f74d10e-kube-api-access-7w88s\") pod \"payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs\" (UID: \"6530b7d2-6dc7-4792-a774-61673f74d10e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"
Sep 11 16:42:51.842675 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:51.842527    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6530b7d2-6dc7-4792-a774-61673f74d10e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs\" (UID: \"6530b7d2-6dc7-4792-a774-61673f74d10e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"
Sep 11 16:42:51.843188 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:51.843170    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6530b7d2-6dc7-4792-a774-61673f74d10e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs\" (UID: \"6530b7d2-6dc7-4792-a774-61673f74d10e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"
Sep 11 16:42:51.852278 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:51.852256    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w88s\" (UniqueName: \"kubernetes.io/projected/6530b7d2-6dc7-4792-a774-61673f74d10e-kube-api-access-7w88s\") pod \"payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs\" (UID: \"6530b7d2-6dc7-4792-a774-61673f74d10e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"
Sep 11 16:42:52.018200 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:52.018170    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"
Sep 11 16:42:52.293955 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:52.293740    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"]
Sep 11 16:42:52.297208 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:42:52.297180    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6530b7d2_6dc7_4792_a774_61673f74d10e.slice/crio-0c3d74addc9affac9a6321d34b256c0eaff3900035be1bb1bbd5cb343ce7d9af WatchSource:0}: Error finding container 0c3d74addc9affac9a6321d34b256c0eaff3900035be1bb1bbd5cb343ce7d9af: Status 404 returned error can't find the container with id 0c3d74addc9affac9a6321d34b256c0eaff3900035be1bb1bbd5cb343ce7d9af
Sep 11 16:42:52.690965 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:52.690852    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs" event={"ID":"6530b7d2-6dc7-4792-a774-61673f74d10e","Type":"ContainerStarted","Data":"0c3d74addc9affac9a6321d34b256c0eaff3900035be1bb1bbd5cb343ce7d9af"}
Sep 11 16:42:54.706424 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:54.706385    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs" event={"ID":"6530b7d2-6dc7-4792-a774-61673f74d10e","Type":"ContainerStarted","Data":"0ac02a5466cfff20410b5a622c2e357776c46568df4b570df2d9aa4a336057c4"}
Sep 11 16:42:54.706828 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:54.706514    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"
Sep 11 16:42:54.724038 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:54.723981    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs" podStartSLOduration=1.987513403 podStartE2EDuration="3.723962246s" totalImagesPullingTime="1.736448843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:42:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:42:54.722841793 +0000 UTC m=+793.276011718" watchObservedRunningTime="2026-09-11 16:42:54.723962246 +0000 UTC m=+793.277132172"
Sep 11 16:42:55.690432 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.690401    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"]
Sep 11 16:42:55.694165 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.694145    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.696284 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.696260    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-0e8f65d9-kserve-self-signed-certs\""
Sep 11 16:42:55.705208 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.705187    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"]
Sep 11 16:42:55.777237 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.777209    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzv7n\" (UniqueName: \"kubernetes.io/projected/a984c444-0603-4838-acc1-b99ef37c6706-kube-api-access-lzv7n\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.777619 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.777246    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-model-cache\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.777619 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.777269    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a984c444-0603-4838-acc1-b99ef37c6706-tls-certs\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.777619 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.777324    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-home\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.777619 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.777409    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-dshm\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.777619 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.777469    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-tmp-dir\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.878501 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.878459    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzv7n\" (UniqueName: \"kubernetes.io/projected/a984c444-0603-4838-acc1-b99ef37c6706-kube-api-access-lzv7n\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.878501 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.878510    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-model-cache\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.878681 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.878627    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a984c444-0603-4838-acc1-b99ef37c6706-tls-certs\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.878681 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.878668    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-home\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.878797 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.878724    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-dshm\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.878797 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.878760    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-tmp-dir\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.878913 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.878821    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-model-cache\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.879057 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.879031    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-home\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.879146 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.879122    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-tmp-dir\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.881536 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.881507    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-dshm\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.881949 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.881929    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a984c444-0603-4838-acc1-b99ef37c6706-tls-certs\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.889220 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.889199    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzv7n\" (UniqueName: \"kubernetes.io/projected/a984c444-0603-4838-acc1-b99ef37c6706-kube-api-access-lzv7n\") pod \"e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") " pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:55.950096 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:55.950030    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:42:56.003681 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:56.003653    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:42:56.279833 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:56.279799    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"]
Sep 11 16:42:56.282572 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:42:56.281628    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda984c444_0603_4838_acc1_b99ef37c6706.slice/crio-ced3513c6e42a0451926a010b277348ac84174d82fbeafde022f9d302a06725d WatchSource:0}: Error finding container ced3513c6e42a0451926a010b277348ac84174d82fbeafde022f9d302a06725d: Status 404 returned error can't find the container with id ced3513c6e42a0451926a010b277348ac84174d82fbeafde022f9d302a06725d
Sep 11 16:42:56.719690 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:56.719608    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s" event={"ID":"a984c444-0603-4838-acc1-b99ef37c6706","Type":"ContainerStarted","Data":"90c4b704dc8e9485c9d694d06b71ada9ec7d372d10f615a8036e3b6b87250a3e"}
Sep 11 16:42:56.719690 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:56.719645    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s" event={"ID":"a984c444-0603-4838-acc1-b99ef37c6706","Type":"ContainerStarted","Data":"ced3513c6e42a0451926a010b277348ac84174d82fbeafde022f9d302a06725d"}
Sep 11 16:42:56.743423 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:42:56.743353    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s" podStartSLOduration=1.7433387150000001 podStartE2EDuration="1.743338715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:42:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:42:56.742300204 +0000 UTC m=+795.295470139" watchObservedRunningTime="2026-09-11 16:42:56.743338715 +0000 UTC m=+795.296508640"
Sep 11 16:43:05.713007 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:05.712964    2545 patch_prober.go:28] interesting pod/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 16:43:05.713478 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:05.713024    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk" podUID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 16:43:05.714705 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:05.714677    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"
Sep 11 16:43:06.004696 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:06.004658    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:43:06.004907 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:06.004718    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:43:06.022460 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:06.022435    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:43:06.789662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:06.789626    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:43:07.783607 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:07.783575    2545 generic.go:356] "Generic (PLEG): container finished" podID="6d250f02-6408-42a9-8183-7b7bbdd723f1" containerID="f4780dde6baf8b5dec3b007986d2537f12b64fc2dbc53718a13ecbb3b0261e72" exitCode=0
Sep 11 16:43:07.783835 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:07.783643    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5" event={"ID":"6d250f02-6408-42a9-8183-7b7bbdd723f1","Type":"ContainerDied","Data":"f4780dde6baf8b5dec3b007986d2537f12b64fc2dbc53718a13ecbb3b0261e72"}
Sep 11 16:43:07.784115 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:07.784094    2545 scope.go:117] "RemoveContainer" containerID="f4780dde6baf8b5dec3b007986d2537f12b64fc2dbc53718a13ecbb3b0261e72"
Sep 11 16:43:07.785260 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:07.785232    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:07.785344 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:07.785301    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:07.785344 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:07.785336    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:07.785463 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:07.785446    2545 generic.go:356] "Generic (PLEG): container finished" podID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerID="955443170f1b93c43b4d7185eded13252c40767f7386280dd06b84b2580bcbfc" exitCode=0
Sep 11 16:43:07.785529 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:07.785513    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk" event={"ID":"690ceb1a-1b49-48eb-9d51-c18a377cbb4b","Type":"ContainerDied","Data":"955443170f1b93c43b4d7185eded13252c40767f7386280dd06b84b2580bcbfc"}
Sep 11 16:43:07.785928 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:07.785913    2545 scope.go:117] "RemoveContainer" containerID="955443170f1b93c43b4d7185eded13252c40767f7386280dd06b84b2580bcbfc"
Sep 11 16:43:07.786779 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:07.786756    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:07.786894 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:07.786810    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:07.786894 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:07.786840    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:08.797587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:08.797538    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk" event={"ID":"690ceb1a-1b49-48eb-9d51-c18a377cbb4b","Type":"ContainerStarted","Data":"c39a73a7a0f6c187b414bd98298e302d2ba59730c4e066b11241b38b66b2a923"}
Sep 11 16:43:08.801661 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:08.801609    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5" event={"ID":"6d250f02-6408-42a9-8183-7b7bbdd723f1","Type":"ContainerStarted","Data":"01473437f7fb834e1f566a4fe4b9b6181d3d3c2401b3580d8defbcb29ebffcce"}
Sep 11 16:43:09.376179 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:09.375729    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:43:09.376179 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:09.375852    2545 patch_prober.go:28] interesting pod/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 11 16:43:09.376179 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:09.375927    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5" podUID="6d250f02-6408-42a9-8183-7b7bbdd723f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 11 16:43:09.542073 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:09.542035    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:43:09.542238 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:09.542157    2545 patch_prober.go:28] interesting pod/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 16:43:09.542238 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:09.542199    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk" podUID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 16:43:10.375907 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:10.375838    2545 patch_prober.go:28] interesting pod/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 11 16:43:10.376368 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:10.375928    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5" podUID="6d250f02-6408-42a9-8183-7b7bbdd723f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 11 16:43:10.542817 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:10.542781    2545 patch_prober.go:28] interesting pod/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 16:43:10.543003 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:10.542834    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk" podUID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 16:43:11.376545 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:11.376502    2545 patch_prober.go:28] interesting pod/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 11 16:43:11.377143 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:11.377109    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5" podUID="6d250f02-6408-42a9-8183-7b7bbdd723f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 11 16:43:11.542908 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:11.542853    2545 patch_prober.go:28] interesting pod/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 16:43:11.543078 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:11.542939    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk" podUID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 16:43:12.376927 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.376472    2545 patch_prober.go:28] interesting pod/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 11 16:43:12.376927 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.376535    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5" podUID="6d250f02-6408-42a9-8183-7b7bbdd723f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 11 16:43:12.542387 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.542347    2545 patch_prober.go:28] interesting pod/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 16:43:12.542540 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.542427    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk" podUID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 16:43:12.685685 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.685595    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"]
Sep 11 16:43:12.692333 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.692307    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.697458 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.697437    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-587a17-openshift-default-dockercfg-th468\""
Sep 11 16:43:12.733716 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.733673    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"]
Sep 11 16:43:12.853757 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.853727    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-credential-socket\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.853757 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.853758    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-workload-certs\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.853992 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.853881    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1997c155-f9e6-44be-91ce-eafa4931d09a-istiod-ca-cert\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.853992 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.853926    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb5v5\" (UniqueName: \"kubernetes.io/projected/1997c155-f9e6-44be-91ce-eafa4931d09a-kube-api-access-zb5v5\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.853992 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.853979    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-podinfo\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.854090 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.854020    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-data\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.854090 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.854048    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-workload-socket\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.854090 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.854074    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-token\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.854178 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.854092    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-envoy\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.955543 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.955443    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1997c155-f9e6-44be-91ce-eafa4931d09a-istiod-ca-cert\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.955543 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.955495    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb5v5\" (UniqueName: \"kubernetes.io/projected/1997c155-f9e6-44be-91ce-eafa4931d09a-kube-api-access-zb5v5\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.955543 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.955528    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-podinfo\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.955814 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.955561    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-data\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.955814 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.955580    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-workload-socket\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.955814 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.955605    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-token\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.955814 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.955623    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-envoy\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.955814 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.955656    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-credential-socket\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.955814 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.955669    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-workload-certs\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.956148 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.956000    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-data\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.956148 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.956054    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-workload-certs\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.956367 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.956342    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-credential-socket\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.956436 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.956361    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1997c155-f9e6-44be-91ce-eafa4931d09a-istiod-ca-cert\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.956591 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.956568    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-workload-socket\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.958224 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.958184    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-envoy\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.958855 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.958830    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-podinfo\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.970930 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.970887    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-token\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:12.971508 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:12.971486    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb5v5\" (UniqueName: \"kubernetes.io/projected/1997c155-f9e6-44be-91ce-eafa4931d09a-kube-api-access-zb5v5\") pod \"e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") " pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:13.009581 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:13.009316    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:13.328743 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:13.328701    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"]
Sep 11 16:43:13.335763 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:43:13.335736    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1997c155_f9e6_44be_91ce_eafa4931d09a.slice/crio-9514eb8dae9b3b4c7649bd248a035af78815e94aec7ea981143bd7e45dcf3cd9 WatchSource:0}: Error finding container 9514eb8dae9b3b4c7649bd248a035af78815e94aec7ea981143bd7e45dcf3cd9: Status 404 returned error can't find the container with id 9514eb8dae9b3b4c7649bd248a035af78815e94aec7ea981143bd7e45dcf3cd9
Sep 11 16:43:13.338176 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:13.338133    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:13.338283 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:13.338250    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:13.338323 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:13.338313    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:13.381650 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:13.381238    2545 patch_prober.go:28] interesting pod/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 11 16:43:13.381650 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:13.381300    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5" podUID="6d250f02-6408-42a9-8183-7b7bbdd723f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 11 16:43:13.542628 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:13.542583    2545 patch_prober.go:28] interesting pod/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 16:43:13.542825 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:13.542648    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk" podUID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 16:43:13.846214 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:13.846171    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b" event={"ID":"1997c155-f9e6-44be-91ce-eafa4931d09a","Type":"ContainerStarted","Data":"90a9410e8458a914fca5f577268cb898cb024be63b815372dcda870f1900c29a"}
Sep 11 16:43:13.846390 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:13.846243    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b" event={"ID":"1997c155-f9e6-44be-91ce-eafa4931d09a","Type":"ContainerStarted","Data":"9514eb8dae9b3b4c7649bd248a035af78815e94aec7ea981143bd7e45dcf3cd9"}
Sep 11 16:43:13.891148 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:13.891093    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b" podStartSLOduration=1.891076797 podStartE2EDuration="1.891076797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:43:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:43:13.887799617 +0000 UTC m=+812.440969543" watchObservedRunningTime="2026-09-11 16:43:13.891076797 +0000 UTC m=+812.444246723"
Sep 11 16:43:14.009957 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:14.009922    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:14.015400 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:14.015372    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:14.380235 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:14.380204    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:43:14.380500 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:14.380481    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:43:14.381131 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:14.381115    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:43:14.545489 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:14.545465    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:43:14.545853 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:14.545783    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:43:14.546606 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:14.546587    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:43:14.852563 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:14.852531    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:14.853420 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:14.853398    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:16.875535 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:16.875503    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:43:20.731104 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:20.731070    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx"]
Sep 11 16:43:20.735290 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:20.735270    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx"
Sep 11 16:43:20.737733 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:20.737709    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:43:20.738235 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:20.738219    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:43:20.745198 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:20.745179    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx"]
Sep 11 16:43:20.833153 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:20.833100    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f783b5-ff04-4651-aa18-5bfa5c2e6a80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx\" (UID: \"08f783b5-ff04-4651-aa18-5bfa5c2e6a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx"
Sep 11 16:43:20.933682 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:20.933652    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f783b5-ff04-4651-aa18-5bfa5c2e6a80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx\" (UID: \"08f783b5-ff04-4651-aa18-5bfa5c2e6a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx"
Sep 11 16:43:20.934282 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:20.934265    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f783b5-ff04-4651-aa18-5bfa5c2e6a80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx\" (UID: \"08f783b5-ff04-4651-aa18-5bfa5c2e6a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx"
Sep 11 16:43:21.047758 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:21.047729    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx"
Sep 11 16:43:21.329416 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:21.329228    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx"]
Sep 11 16:43:21.331812 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:43:21.331783    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08f783b5_ff04_4651_aa18_5bfa5c2e6a80.slice/crio-637bf121b3c76d4af2843d2a8f0dc6fb490e796f7019f2e7f52778401b2e1531 WatchSource:0}: Error finding container 637bf121b3c76d4af2843d2a8f0dc6fb490e796f7019f2e7f52778401b2e1531: Status 404 returned error can't find the container with id 637bf121b3c76d4af2843d2a8f0dc6fb490e796f7019f2e7f52778401b2e1531
Sep 11 16:43:21.891147 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:21.891113    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx" event={"ID":"08f783b5-ff04-4651-aa18-5bfa5c2e6a80","Type":"ContainerStarted","Data":"637bf121b3c76d4af2843d2a8f0dc6fb490e796f7019f2e7f52778401b2e1531"}
Sep 11 16:43:23.904156 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:23.904111    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx" event={"ID":"08f783b5-ff04-4651-aa18-5bfa5c2e6a80","Type":"ContainerStarted","Data":"b6a6ff8db2390a1ae825fbde8d5c13e02b69dcee8bdb2bc94a466a4332063e68"}
Sep 11 16:43:23.967411 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:23.967318    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx" podStartSLOduration=1.8976891280000001 podStartE2EDuration="3.967303289s" totalImagesPullingTime="2.069614161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:43:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:43:23.966769287 +0000 UTC m=+822.519939212" watchObservedRunningTime="2026-09-11 16:43:23.967303289 +0000 UTC m=+822.520473214"
Sep 11 16:43:24.910469 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:24.910433    2545 generic.go:356] "Generic (PLEG): container finished" podID="08f783b5-ff04-4651-aa18-5bfa5c2e6a80" containerID="b6a6ff8db2390a1ae825fbde8d5c13e02b69dcee8bdb2bc94a466a4332063e68" exitCode=0
Sep 11 16:43:24.910806 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:24.910518    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx" event={"ID":"08f783b5-ff04-4651-aa18-5bfa5c2e6a80","Type":"ContainerDied","Data":"b6a6ff8db2390a1ae825fbde8d5c13e02b69dcee8bdb2bc94a466a4332063e68"}
Sep 11 16:43:26.069888 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:26.069847    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx"
Sep 11 16:43:26.206081 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:26.206000    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f783b5-ff04-4651-aa18-5bfa5c2e6a80-maas-api-service-ca\") pod \"08f783b5-ff04-4651-aa18-5bfa5c2e6a80\" (UID: \"08f783b5-ff04-4651-aa18-5bfa5c2e6a80\") "
Sep 11 16:43:26.206367 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:26.206342    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08f783b5-ff04-4651-aa18-5bfa5c2e6a80-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "08f783b5-ff04-4651-aa18-5bfa5c2e6a80" (UID: "08f783b5-ff04-4651-aa18-5bfa5c2e6a80"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:43:26.307097 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:26.307061    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f783b5-ff04-4651-aa18-5bfa5c2e6a80-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:43:26.923017 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:26.922987    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx"
Sep 11 16:43:26.923201 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:26.923035    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx" event={"ID":"08f783b5-ff04-4651-aa18-5bfa5c2e6a80","Type":"ContainerDied","Data":"637bf121b3c76d4af2843d2a8f0dc6fb490e796f7019f2e7f52778401b2e1531"}
Sep 11 16:43:26.923201 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:26.923093    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="637bf121b3c76d4af2843d2a8f0dc6fb490e796f7019f2e7f52778401b2e1531"
Sep 11 16:43:31.575669 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:31.575634    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:43:32.543893 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:32.543850    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx"]
Sep 11 16:43:32.547309 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:32.547289    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-845e25d3-s9vjx"]
Sep 11 16:43:32.686712 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:32.686682    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"]
Sep 11 16:43:32.687055 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:32.686973    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk" podUID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerName="istio-proxy" containerID="cri-o://c39a73a7a0f6c187b414bd98298e302d2ba59730c4e066b11241b38b66b2a923" gracePeriod=30
Sep 11 16:43:33.739660 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.739627    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"]
Sep 11 16:43:33.742016 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.740319    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08f783b5-ff04-4651-aa18-5bfa5c2e6a80" containerName="revoke-keys"
Sep 11 16:43:33.742016 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.740339    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="08f783b5-ff04-4651-aa18-5bfa5c2e6a80" containerName="revoke-keys"
Sep 11 16:43:33.742016 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.740412    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="08f783b5-ff04-4651-aa18-5bfa5c2e6a80" containerName="revoke-keys"
Sep 11 16:43:33.743271 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.743251    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.746110 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.746089    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-983a9aee-gw-openshift-default-dockercfg-2sd7j\""
Sep 11 16:43:33.759389 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.759366    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"]
Sep 11 16:43:33.775206 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.775184    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-credential-socket\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.775336 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.775211    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-podinfo\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.775336 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.775234    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-workload-certs\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.775464 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.775340    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c84535e-90d4-4066-88ac-67b11ccc89fd-istiod-ca-cert\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.775464 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.775377    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-data\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.775464 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.775396    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-token\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.775464 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.775428    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-envoy\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.775676 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.775466    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pljjj\" (UniqueName: \"kubernetes.io/projected/1c84535e-90d4-4066-88ac-67b11ccc89fd-kube-api-access-pljjj\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.775676 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.775568    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-workload-socket\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.877184 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.877153    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-workload-socket\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.877361 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.877190    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-credential-socket\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.877361 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.877211    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-podinfo\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.877361 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.877242    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-workload-certs\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.877361 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.877315    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c84535e-90d4-4066-88ac-67b11ccc89fd-istiod-ca-cert\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.877361 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.877355    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-data\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.877634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.877381    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-token\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.877634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.877419    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-envoy\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.877634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.877475    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pljjj\" (UniqueName: \"kubernetes.io/projected/1c84535e-90d4-4066-88ac-67b11ccc89fd-kube-api-access-pljjj\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.877634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.877588    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-credential-socket\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.877853 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.877639    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-workload-socket\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.877853 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.877699    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-workload-certs\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.878041 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.878014    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-data\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.878550 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.878528    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c84535e-90d4-4066-88ac-67b11ccc89fd-istiod-ca-cert\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.879974 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.879953    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-envoy\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.880708 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.880690    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-podinfo\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.889929 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.889911    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pljjj\" (UniqueName: \"kubernetes.io/projected/1c84535e-90d4-4066-88ac-67b11ccc89fd-kube-api-access-pljjj\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.890647 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.890631    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-token\") pod \"e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") " pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:33.939267 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.939158    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08f783b5-ff04-4651-aa18-5bfa5c2e6a80" path="/var/lib/kubelet/pods/08f783b5-ff04-4651-aa18-5bfa5c2e6a80/volumes"
Sep 11 16:43:33.960557 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.960535    2545 generic.go:356] "Generic (PLEG): container finished" podID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerID="90a9410e8458a914fca5f577268cb898cb024be63b815372dcda870f1900c29a" exitCode=0
Sep 11 16:43:33.960668 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.960617    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b" event={"ID":"1997c155-f9e6-44be-91ce-eafa4931d09a","Type":"ContainerDied","Data":"90a9410e8458a914fca5f577268cb898cb024be63b815372dcda870f1900c29a"}
Sep 11 16:43:33.961030 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.961016    2545 scope.go:117] "RemoveContainer" containerID="90a9410e8458a914fca5f577268cb898cb024be63b815372dcda870f1900c29a"
Sep 11 16:43:33.962171 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.962145    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:33.962238 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.962211    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:33.962272 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:33.962243    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:34.057069 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:34.057046    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:34.349253 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:34.349230    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"]
Sep 11 16:43:34.351795 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:43:34.351760    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c84535e_90d4_4066_88ac_67b11ccc89fd.slice/crio-2c8d4c7deada759b6f9105a569ba10176a0e86366bf68153a1449fb359b82fa0 WatchSource:0}: Error finding container 2c8d4c7deada759b6f9105a569ba10176a0e86366bf68153a1449fb359b82fa0: Status 404 returned error can't find the container with id 2c8d4c7deada759b6f9105a569ba10176a0e86366bf68153a1449fb359b82fa0
Sep 11 16:43:34.354199 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:34.354169    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:34.354296 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:34.354243    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:34.354296 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:34.354286    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:34.968496 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:34.968466    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b" event={"ID":"1997c155-f9e6-44be-91ce-eafa4931d09a","Type":"ContainerStarted","Data":"6beb42224c8bb05d59a418b882a734216700bca0bd5323858efb16344cc076b4"}
Sep 11 16:43:34.970066 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:34.970039    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc" event={"ID":"1c84535e-90d4-4066-88ac-67b11ccc89fd","Type":"ContainerStarted","Data":"3d722abe2608d32be024c86b89a3a482d1ba31644298112b3c5d3167c82a1a8b"}
Sep 11 16:43:34.970179 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:34.970091    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc" event={"ID":"1c84535e-90d4-4066-88ac-67b11ccc89fd","Type":"ContainerStarted","Data":"2c8d4c7deada759b6f9105a569ba10176a0e86366bf68153a1449fb359b82fa0"}
Sep 11 16:43:35.010156 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:35.010115    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:35.010364 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:35.010339    2545 patch_prober.go:28] interesting pod/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 11 16:43:35.010453 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:35.010386    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b" podUID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 11 16:43:35.040009 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:35.039961    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc" podStartSLOduration=2.039945973 podStartE2EDuration="2.039945973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:43:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:43:35.037334959 +0000 UTC m=+833.590504918" watchObservedRunningTime="2026-09-11 16:43:35.039945973 +0000 UTC m=+833.593115899"
Sep 11 16:43:35.057834 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:35.057795    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:35.062993 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:35.062972    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:35.975188 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:35.975162    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:35.980279 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:35.980252    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:36.010558 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:36.010533    2545 patch_prober.go:28] interesting pod/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 11 16:43:36.010687 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:36.010580    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b" podUID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 11 16:43:37.010425 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:37.010390    2545 patch_prober.go:28] interesting pod/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 11 16:43:37.010794 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:37.010443    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b" podUID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 11 16:43:37.960931 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:37.960842    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:43:38.002755 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.001825    2545 generic.go:356] "Generic (PLEG): container finished" podID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerID="c39a73a7a0f6c187b414bd98298e302d2ba59730c4e066b11241b38b66b2a923" exitCode=0
Sep 11 16:43:38.002755 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.001996    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk" event={"ID":"690ceb1a-1b49-48eb-9d51-c18a377cbb4b","Type":"ContainerDied","Data":"c39a73a7a0f6c187b414bd98298e302d2ba59730c4e066b11241b38b66b2a923"}
Sep 11 16:43:38.002755 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.002072    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk" event={"ID":"690ceb1a-1b49-48eb-9d51-c18a377cbb4b","Type":"ContainerDied","Data":"ad62ff3812ea53def5d4d758601f89edaac26c2fa2aa7ec1147ed23aedcf975e"}
Sep 11 16:43:38.002755 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.002109    2545 scope.go:117] "RemoveContainer" containerID="c39a73a7a0f6c187b414bd98298e302d2ba59730c4e066b11241b38b66b2a923"
Sep 11 16:43:38.002755 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.002296    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"
Sep 11 16:43:38.010631 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.010599    2545 patch_prober.go:28] interesting pod/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 11 16:43:38.011025 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.010676    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b" podUID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 11 16:43:38.016637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.016614    2545 scope.go:117] "RemoveContainer" containerID="955443170f1b93c43b4d7185eded13252c40767f7386280dd06b84b2580bcbfc"
Sep 11 16:43:38.025157 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.025134    2545 scope.go:117] "RemoveContainer" containerID="c39a73a7a0f6c187b414bd98298e302d2ba59730c4e066b11241b38b66b2a923"
Sep 11 16:43:38.025569 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:43:38.025550    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c39a73a7a0f6c187b414bd98298e302d2ba59730c4e066b11241b38b66b2a923\": container with ID starting with c39a73a7a0f6c187b414bd98298e302d2ba59730c4e066b11241b38b66b2a923 not found: ID does not exist" containerID="c39a73a7a0f6c187b414bd98298e302d2ba59730c4e066b11241b38b66b2a923"
Sep 11 16:43:38.025660 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.025580    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c39a73a7a0f6c187b414bd98298e302d2ba59730c4e066b11241b38b66b2a923"} err="failed to get container status \"c39a73a7a0f6c187b414bd98298e302d2ba59730c4e066b11241b38b66b2a923\": rpc error: code = NotFound desc = could not find container \"c39a73a7a0f6c187b414bd98298e302d2ba59730c4e066b11241b38b66b2a923\": container with ID starting with c39a73a7a0f6c187b414bd98298e302d2ba59730c4e066b11241b38b66b2a923 not found: ID does not exist"
Sep 11 16:43:38.025660 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.025603    2545 scope.go:117] "RemoveContainer" containerID="955443170f1b93c43b4d7185eded13252c40767f7386280dd06b84b2580bcbfc"
Sep 11 16:43:38.025965 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:43:38.025938    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"955443170f1b93c43b4d7185eded13252c40767f7386280dd06b84b2580bcbfc\": container with ID starting with 955443170f1b93c43b4d7185eded13252c40767f7386280dd06b84b2580bcbfc not found: ID does not exist" containerID="955443170f1b93c43b4d7185eded13252c40767f7386280dd06b84b2580bcbfc"
Sep 11 16:43:38.026084 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.025963    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"955443170f1b93c43b4d7185eded13252c40767f7386280dd06b84b2580bcbfc"} err="failed to get container status \"955443170f1b93c43b4d7185eded13252c40767f7386280dd06b84b2580bcbfc\": rpc error: code = NotFound desc = could not find container \"955443170f1b93c43b4d7185eded13252c40767f7386280dd06b84b2580bcbfc\": container with ID starting with 955443170f1b93c43b4d7185eded13252c40767f7386280dd06b84b2580bcbfc not found: ID does not exist"
Sep 11 16:43:38.028289 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.028269    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-workload-socket\") pod \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") "
Sep 11 16:43:38.028387 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.028314    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-credential-socket\") pod \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") "
Sep 11 16:43:38.028387 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.028348    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-data\") pod \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") "
Sep 11 16:43:38.028387 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.028370    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-workload-certs\") pod \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") "
Sep 11 16:43:38.028565 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.028398    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5d2l\" (UniqueName: \"kubernetes.io/projected/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-kube-api-access-k5d2l\") pod \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") "
Sep 11 16:43:38.028565 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.028420    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-token\") pod \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") "
Sep 11 16:43:38.028565 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.028470    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-envoy\") pod \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") "
Sep 11 16:43:38.028565 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.028490    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "690ceb1a-1b49-48eb-9d51-c18a377cbb4b" (UID: "690ceb1a-1b49-48eb-9d51-c18a377cbb4b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:43:38.028565 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.028545    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istiod-ca-cert\") pod \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") "
Sep 11 16:43:38.028833 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.028618    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-podinfo\") pod \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\" (UID: \"690ceb1a-1b49-48eb-9d51-c18a377cbb4b\") "
Sep 11 16:43:38.028833 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.028628    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "690ceb1a-1b49-48eb-9d51-c18a377cbb4b" (UID: "690ceb1a-1b49-48eb-9d51-c18a377cbb4b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:43:38.028833 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.028643    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-data" (OuterVolumeSpecName: "istio-data") pod "690ceb1a-1b49-48eb-9d51-c18a377cbb4b" (UID: "690ceb1a-1b49-48eb-9d51-c18a377cbb4b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:43:38.028833 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.028652    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "690ceb1a-1b49-48eb-9d51-c18a377cbb4b" (UID: "690ceb1a-1b49-48eb-9d51-c18a377cbb4b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:43:38.029063 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.028939    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "690ceb1a-1b49-48eb-9d51-c18a377cbb4b" (UID: "690ceb1a-1b49-48eb-9d51-c18a377cbb4b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:43:38.029200 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.029180    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:43:38.029265 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.029205    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:43:38.029265 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.029218    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:43:38.029265 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.029231    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:43:38.029265 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.029244    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:43:38.031089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.031066    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "690ceb1a-1b49-48eb-9d51-c18a377cbb4b" (UID: "690ceb1a-1b49-48eb-9d51-c18a377cbb4b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:43:38.031218 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.031084    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-kube-api-access-k5d2l" (OuterVolumeSpecName: "kube-api-access-k5d2l") pod "690ceb1a-1b49-48eb-9d51-c18a377cbb4b" (UID: "690ceb1a-1b49-48eb-9d51-c18a377cbb4b"). InnerVolumeSpecName "kube-api-access-k5d2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:43:38.031332 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.031315    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "690ceb1a-1b49-48eb-9d51-c18a377cbb4b" (UID: "690ceb1a-1b49-48eb-9d51-c18a377cbb4b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:43:38.032339 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.032316    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-token" (OuterVolumeSpecName: "istio-token") pod "690ceb1a-1b49-48eb-9d51-c18a377cbb4b" (UID: "690ceb1a-1b49-48eb-9d51-c18a377cbb4b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:43:38.130420 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.130390    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:43:38.130420 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.130418    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5d2l\" (UniqueName: \"kubernetes.io/projected/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-kube-api-access-k5d2l\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:43:38.130582 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.130427    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:43:38.130582 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.130437    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/690ceb1a-1b49-48eb-9d51-c18a377cbb4b-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:43:38.325780 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.325746    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"]
Sep 11 16:43:38.329791 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:38.329766    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-845e25d3-openshift-default-84b77fd5b-rjdkk"]
Sep 11 16:43:39.011094 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:39.011064    2545 patch_prober.go:28] interesting pod/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 11 16:43:39.011473 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:39.011112    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b" podUID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 11 16:43:39.937547 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:39.937506    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" path="/var/lib/kubelet/pods/690ceb1a-1b49-48eb-9d51-c18a377cbb4b/volumes"
Sep 11 16:43:40.013490 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:40.013464    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:40.013940 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:40.013839    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:40.014485 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:40.014469    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:43:52.017018 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:52.016967    2545 patch_prober.go:28] interesting pod/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 16:43:52.017478 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:52.017044    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc" podUID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 16:43:52.753176 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:52.753147    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:43:53.095211 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:53.095136    2545 generic.go:356] "Generic (PLEG): container finished" podID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerID="3d722abe2608d32be024c86b89a3a482d1ba31644298112b3c5d3167c82a1a8b" exitCode=0
Sep 11 16:43:53.095211 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:53.095189    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc" event={"ID":"1c84535e-90d4-4066-88ac-67b11ccc89fd","Type":"ContainerDied","Data":"3d722abe2608d32be024c86b89a3a482d1ba31644298112b3c5d3167c82a1a8b"}
Sep 11 16:43:53.095592 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:53.095515    2545 scope.go:117] "RemoveContainer" containerID="3d722abe2608d32be024c86b89a3a482d1ba31644298112b3c5d3167c82a1a8b"
Sep 11 16:43:53.096579 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:53.096552    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:53.096637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:53.096627    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:53.096674 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:53.096663    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:43:54.105545 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:54.105513    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc" event={"ID":"1c84535e-90d4-4066-88ac-67b11ccc89fd","Type":"ContainerStarted","Data":"48f42cbb01f263589e8ee1750fbc3ddc4b83734b37ca5dabf056bd31f87c80f7"}
Sep 11 16:43:55.058080 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:55.058044    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:43:55.058248 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:55.058191    2545 patch_prober.go:28] interesting pod/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 11 16:43:55.058291 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:55.058239    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc" podUID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 11 16:43:56.057817 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:56.057786    2545 patch_prober.go:28] interesting pod/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 11 16:43:56.058182 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:56.057844    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc" podUID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 11 16:43:57.058096 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:57.058061    2545 patch_prober.go:28] interesting pod/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 11 16:43:57.058453 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:57.058125    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc" podUID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 11 16:43:58.057423 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:58.057386    2545 patch_prober.go:28] interesting pod/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 11 16:43:58.057591 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:58.057444    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc" podUID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 11 16:43:59.057692 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:59.057660    2545 patch_prober.go:28] interesting pod/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 11 16:43:59.058095 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:43:59.057713    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc" podUID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 11 16:44:00.139840 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:00.139811    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:44:00.140254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:00.140098    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:44:00.140787 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:00.140772    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:44:01.459194 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.459166    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb"]
Sep 11 16:44:01.459615 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.459601    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerName="istio-proxy"
Sep 11 16:44:01.459658 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.459618    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerName="istio-proxy"
Sep 11 16:44:01.459658 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.459631    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerName="istio-proxy"
Sep 11 16:44:01.459658 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.459637    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerName="istio-proxy"
Sep 11 16:44:01.459755 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.459713    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerName="istio-proxy"
Sep 11 16:44:01.459755 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.459722    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="690ceb1a-1b49-48eb-9d51-c18a377cbb4b" containerName="istio-proxy"
Sep 11 16:44:01.463037 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.463023    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb"
Sep 11 16:44:01.465220 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.465194    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:44:01.465476 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.465461    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:44:01.473796 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.473771    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb"]
Sep 11 16:44:01.559532 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.559500    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54c33161-e2fd-4151-8469-021fa0e73a5d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb\" (UID: \"54c33161-e2fd-4151-8469-021fa0e73a5d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb"
Sep 11 16:44:01.660312 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.660282    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54c33161-e2fd-4151-8469-021fa0e73a5d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb\" (UID: \"54c33161-e2fd-4151-8469-021fa0e73a5d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb"
Sep 11 16:44:01.660884 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.660843    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54c33161-e2fd-4151-8469-021fa0e73a5d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb\" (UID: \"54c33161-e2fd-4151-8469-021fa0e73a5d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb"
Sep 11 16:44:01.773759 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:01.773725    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb"
Sep 11 16:44:02.048739 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:02.048719    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb"]
Sep 11 16:44:02.051093 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:44:02.051066    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54c33161_e2fd_4151_8469_021fa0e73a5d.slice/crio-04095d2f55967738c7b79ea3f8f0454bb5a3d95b791c467a42b98ee2f9bead19 WatchSource:0}: Error finding container 04095d2f55967738c7b79ea3f8f0454bb5a3d95b791c467a42b98ee2f9bead19: Status 404 returned error can't find the container with id 04095d2f55967738c7b79ea3f8f0454bb5a3d95b791c467a42b98ee2f9bead19
Sep 11 16:44:02.154134 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:02.154102    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb" event={"ID":"54c33161-e2fd-4151-8469-021fa0e73a5d","Type":"ContainerStarted","Data":"04095d2f55967738c7b79ea3f8f0454bb5a3d95b791c467a42b98ee2f9bead19"}
Sep 11 16:44:03.160822 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:03.160790    2545 generic.go:356] "Generic (PLEG): container finished" podID="54c33161-e2fd-4151-8469-021fa0e73a5d" containerID="b928e64eaf00153edfccd7a89cd2960d084f1bcf6c56161df89ed343acc3e18c" exitCode=0
Sep 11 16:44:03.161250 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:03.160843    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb" event={"ID":"54c33161-e2fd-4151-8469-021fa0e73a5d","Type":"ContainerDied","Data":"b928e64eaf00153edfccd7a89cd2960d084f1bcf6c56161df89ed343acc3e18c"}
Sep 11 16:44:04.298760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:04.298738    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb"
Sep 11 16:44:04.386668 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:04.386641    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54c33161-e2fd-4151-8469-021fa0e73a5d-maas-api-service-ca\") pod \"54c33161-e2fd-4151-8469-021fa0e73a5d\" (UID: \"54c33161-e2fd-4151-8469-021fa0e73a5d\") "
Sep 11 16:44:04.387040 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:04.387012    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54c33161-e2fd-4151-8469-021fa0e73a5d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "54c33161-e2fd-4151-8469-021fa0e73a5d" (UID: "54c33161-e2fd-4151-8469-021fa0e73a5d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:44:04.488387 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:04.488363    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54c33161-e2fd-4151-8469-021fa0e73a5d-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:05.173507 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:05.173479    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb"
Sep 11 16:44:05.173507 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:05.173499    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb" event={"ID":"54c33161-e2fd-4151-8469-021fa0e73a5d","Type":"ContainerDied","Data":"04095d2f55967738c7b79ea3f8f0454bb5a3d95b791c467a42b98ee2f9bead19"}
Sep 11 16:44:05.173806 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:05.173541    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04095d2f55967738c7b79ea3f8f0454bb5a3d95b791c467a42b98ee2f9bead19"
Sep 11 16:44:08.405302 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:08.405269    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb"]
Sep 11 16:44:08.412011 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:08.411988    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-983a9aee-bzsbb"]
Sep 11 16:44:08.544932 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:08.544891    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"]
Sep 11 16:44:08.545184 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:08.545132    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc" podUID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerName="istio-proxy" containerID="cri-o://48f42cbb01f263589e8ee1750fbc3ddc4b83734b37ca5dabf056bd31f87c80f7" gracePeriod=30
Sep 11 16:44:09.254348 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.254316    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"]
Sep 11 16:44:09.254777 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.254763    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c33161-e2fd-4151-8469-021fa0e73a5d" containerName="revoke-keys"
Sep 11 16:44:09.254777 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.254779    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c33161-e2fd-4151-8469-021fa0e73a5d" containerName="revoke-keys"
Sep 11 16:44:09.254911 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.254899    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="54c33161-e2fd-4151-8469-021fa0e73a5d" containerName="revoke-keys"
Sep 11 16:44:09.258104 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.258086    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.260305 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.260287    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-83625e23-openshift-default-dockercfg-7w8tz\""
Sep 11 16:44:09.271840 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.271820    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"]
Sep 11 16:44:09.332844 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.332820    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-data\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.332998 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.332853    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-credential-socket\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.332998 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.332936    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-workload-certs\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.332998 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.332970    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-workload-socket\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.332998 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.332998    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-token\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.333207 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.333018    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istiod-ca-cert\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.333207 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.333037    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r9hl\" (UniqueName: \"kubernetes.io/projected/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-kube-api-access-7r9hl\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.333207 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.333079    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-envoy\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.333207 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.333099    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-podinfo\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.434204 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.434170    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-envoy\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.434204 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.434208    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-podinfo\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.434644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.434239    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-data\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.434644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.434364    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-credential-socket\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.434644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.434438    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-workload-certs\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.434644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.434506    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-workload-socket\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.434644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.434544    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-token\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.434644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.434566    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-data\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.434644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.434628    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istiod-ca-cert\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.435041 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.434660    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r9hl\" (UniqueName: \"kubernetes.io/projected/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-kube-api-access-7r9hl\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.435041 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.434705    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-credential-socket\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.435041 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.434760    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-workload-certs\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.435041 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.435022    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-workload-socket\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.435302 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.435283    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istiod-ca-cert\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.436878 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.436838    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-podinfo\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.437210 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.437190    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-envoy\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.452552 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.452529    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r9hl\" (UniqueName: \"kubernetes.io/projected/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-kube-api-access-7r9hl\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.452755 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.452736    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-token\") pod \"e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") " pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.568159 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.568083    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:09.856557 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.856523    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"]
Sep 11 16:44:09.856684 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:44:09.856526    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb6b7741_2fd4_48da_adc6_0cd3b0ff46ca.slice/crio-b20adc8de95dd7270f4b4566c596a1c17bce062e418b169c3b50d5c9125dfe6d WatchSource:0}: Error finding container b20adc8de95dd7270f4b4566c596a1c17bce062e418b169c3b50d5c9125dfe6d: Status 404 returned error can't find the container with id b20adc8de95dd7270f4b4566c596a1c17bce062e418b169c3b50d5c9125dfe6d
Sep 11 16:44:09.859083 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.859052    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:44:09.859196 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.859128    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:44:09.859196 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.859170    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:44:09.939310 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:09.939273    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54c33161-e2fd-4151-8469-021fa0e73a5d" path="/var/lib/kubelet/pods/54c33161-e2fd-4151-8469-021fa0e73a5d/volumes"
Sep 11 16:44:10.206487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:10.206405    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh" event={"ID":"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca","Type":"ContainerStarted","Data":"f7e31393460de77afb2e0f933b93bcd64560008df764773d19f3bd9284faeac9"}
Sep 11 16:44:10.206487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:10.206456    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh" event={"ID":"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca","Type":"ContainerStarted","Data":"b20adc8de95dd7270f4b4566c596a1c17bce062e418b169c3b50d5c9125dfe6d"}
Sep 11 16:44:10.235885 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:10.235816    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh" podStartSLOduration=1.235800943 podStartE2EDuration="1.235800943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:44:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:10.234080767 +0000 UTC m=+868.787250724" watchObservedRunningTime="2026-09-11 16:44:10.235800943 +0000 UTC m=+868.788970930"
Sep 11 16:44:10.568507 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:10.568475    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:10.573778 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:10.573755    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:11.212801 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:11.212777    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:11.213638 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:11.213618    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:13.811017 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.810954    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:44:13.845666 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.845482    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:44:13.875159 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875122    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-envoy\") pod \"1c84535e-90d4-4066-88ac-67b11ccc89fd\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") "
Sep 11 16:44:13.875159 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875155    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c84535e-90d4-4066-88ac-67b11ccc89fd-istiod-ca-cert\") pod \"1c84535e-90d4-4066-88ac-67b11ccc89fd\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") "
Sep 11 16:44:13.875396 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875177    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-data\") pod \"1c84535e-90d4-4066-88ac-67b11ccc89fd\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") "
Sep 11 16:44:13.875396 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875197    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-credential-socket\") pod \"1c84535e-90d4-4066-88ac-67b11ccc89fd\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") "
Sep 11 16:44:13.875396 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875226    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-workload-socket\") pod \"1c84535e-90d4-4066-88ac-67b11ccc89fd\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") "
Sep 11 16:44:13.875396 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875300    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-token\") pod \"1c84535e-90d4-4066-88ac-67b11ccc89fd\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") "
Sep 11 16:44:13.875396 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875376    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pljjj\" (UniqueName: \"kubernetes.io/projected/1c84535e-90d4-4066-88ac-67b11ccc89fd-kube-api-access-pljjj\") pod \"1c84535e-90d4-4066-88ac-67b11ccc89fd\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") "
Sep 11 16:44:13.875637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875428    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-podinfo\") pod \"1c84535e-90d4-4066-88ac-67b11ccc89fd\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") "
Sep 11 16:44:13.875637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875444    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-data" (OuterVolumeSpecName: "istio-data") pod "1c84535e-90d4-4066-88ac-67b11ccc89fd" (UID: "1c84535e-90d4-4066-88ac-67b11ccc89fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:44:13.875637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875463    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1c84535e-90d4-4066-88ac-67b11ccc89fd" (UID: "1c84535e-90d4-4066-88ac-67b11ccc89fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:44:13.875637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875472    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-workload-certs\") pod \"1c84535e-90d4-4066-88ac-67b11ccc89fd\" (UID: \"1c84535e-90d4-4066-88ac-67b11ccc89fd\") "
Sep 11 16:44:13.875637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875535    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c84535e-90d4-4066-88ac-67b11ccc89fd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1c84535e-90d4-4066-88ac-67b11ccc89fd" (UID: "1c84535e-90d4-4066-88ac-67b11ccc89fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:44:13.875918 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875708    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1c84535e-90d4-4066-88ac-67b11ccc89fd" (UID: "1c84535e-90d4-4066-88ac-67b11ccc89fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:44:13.875918 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875900    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c84535e-90d4-4066-88ac-67b11ccc89fd-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:13.876023 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875924    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:13.876023 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875937    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:13.876023 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875950    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:13.876023 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.875988    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1c84535e-90d4-4066-88ac-67b11ccc89fd" (UID: "1c84535e-90d4-4066-88ac-67b11ccc89fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:44:13.878487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.878461    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-token" (OuterVolumeSpecName: "istio-token") pod "1c84535e-90d4-4066-88ac-67b11ccc89fd" (UID: "1c84535e-90d4-4066-88ac-67b11ccc89fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:44:13.878487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.878468    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c84535e-90d4-4066-88ac-67b11ccc89fd-kube-api-access-pljjj" (OuterVolumeSpecName: "kube-api-access-pljjj") pod "1c84535e-90d4-4066-88ac-67b11ccc89fd" (UID: "1c84535e-90d4-4066-88ac-67b11ccc89fd"). InnerVolumeSpecName "kube-api-access-pljjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:44:13.878751 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.878726    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1c84535e-90d4-4066-88ac-67b11ccc89fd" (UID: "1c84535e-90d4-4066-88ac-67b11ccc89fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:44:13.879248 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.879226    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1c84535e-90d4-4066-88ac-67b11ccc89fd" (UID: "1c84535e-90d4-4066-88ac-67b11ccc89fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:44:13.976854 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.976793    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:13.976854 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.976821    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pljjj\" (UniqueName: \"kubernetes.io/projected/1c84535e-90d4-4066-88ac-67b11ccc89fd-kube-api-access-pljjj\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:13.976854 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.976836    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:13.976854 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.976850    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:13.976854 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:13.976892    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84535e-90d4-4066-88ac-67b11ccc89fd-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:14.230057 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:14.229967    2545 generic.go:356] "Generic (PLEG): container finished" podID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerID="48f42cbb01f263589e8ee1750fbc3ddc4b83734b37ca5dabf056bd31f87c80f7" exitCode=0
Sep 11 16:44:14.230057 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:14.230013    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"
Sep 11 16:44:14.230243 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:14.230052    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc" event={"ID":"1c84535e-90d4-4066-88ac-67b11ccc89fd","Type":"ContainerDied","Data":"48f42cbb01f263589e8ee1750fbc3ddc4b83734b37ca5dabf056bd31f87c80f7"}
Sep 11 16:44:14.230243 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:14.230108    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc" event={"ID":"1c84535e-90d4-4066-88ac-67b11ccc89fd","Type":"ContainerDied","Data":"2c8d4c7deada759b6f9105a569ba10176a0e86366bf68153a1449fb359b82fa0"}
Sep 11 16:44:14.230243 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:14.230132    2545 scope.go:117] "RemoveContainer" containerID="48f42cbb01f263589e8ee1750fbc3ddc4b83734b37ca5dabf056bd31f87c80f7"
Sep 11 16:44:14.240082 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:14.240066    2545 scope.go:117] "RemoveContainer" containerID="3d722abe2608d32be024c86b89a3a482d1ba31644298112b3c5d3167c82a1a8b"
Sep 11 16:44:14.248889 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:14.248820    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"]
Sep 11 16:44:14.255268 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:14.255248    2545 scope.go:117] "RemoveContainer" containerID="48f42cbb01f263589e8ee1750fbc3ddc4b83734b37ca5dabf056bd31f87c80f7"
Sep 11 16:44:14.255743 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:44:14.255720    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48f42cbb01f263589e8ee1750fbc3ddc4b83734b37ca5dabf056bd31f87c80f7\": container with ID starting with 48f42cbb01f263589e8ee1750fbc3ddc4b83734b37ca5dabf056bd31f87c80f7 not found: ID does not exist" containerID="48f42cbb01f263589e8ee1750fbc3ddc4b83734b37ca5dabf056bd31f87c80f7"
Sep 11 16:44:14.255919 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:14.255894    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48f42cbb01f263589e8ee1750fbc3ddc4b83734b37ca5dabf056bd31f87c80f7"} err="failed to get container status \"48f42cbb01f263589e8ee1750fbc3ddc4b83734b37ca5dabf056bd31f87c80f7\": rpc error: code = NotFound desc = could not find container \"48f42cbb01f263589e8ee1750fbc3ddc4b83734b37ca5dabf056bd31f87c80f7\": container with ID starting with 48f42cbb01f263589e8ee1750fbc3ddc4b83734b37ca5dabf056bd31f87c80f7 not found: ID does not exist"
Sep 11 16:44:14.256028 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:14.256017    2545 scope.go:117] "RemoveContainer" containerID="3d722abe2608d32be024c86b89a3a482d1ba31644298112b3c5d3167c82a1a8b"
Sep 11 16:44:14.257469 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:44:14.257180    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d722abe2608d32be024c86b89a3a482d1ba31644298112b3c5d3167c82a1a8b\": container with ID starting with 3d722abe2608d32be024c86b89a3a482d1ba31644298112b3c5d3167c82a1a8b not found: ID does not exist" containerID="3d722abe2608d32be024c86b89a3a482d1ba31644298112b3c5d3167c82a1a8b"
Sep 11 16:44:14.257553 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:14.257488    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d722abe2608d32be024c86b89a3a482d1ba31644298112b3c5d3167c82a1a8b"} err="failed to get container status \"3d722abe2608d32be024c86b89a3a482d1ba31644298112b3c5d3167c82a1a8b\": rpc error: code = NotFound desc = could not find container \"3d722abe2608d32be024c86b89a3a482d1ba31644298112b3c5d3167c82a1a8b\": container with ID starting with 3d722abe2608d32be024c86b89a3a482d1ba31644298112b3c5d3167c82a1a8b not found: ID does not exist"
Sep 11 16:44:14.259952 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:14.259929    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-983a9aee-gw-openshift-default-6cdc478648-z4rjc"]
Sep 11 16:44:15.938243 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:15.938214    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c84535e-90d4-4066-88ac-67b11ccc89fd" path="/var/lib/kubelet/pods/1c84535e-90d4-4066-88ac-67b11ccc89fd/volumes"
Sep 11 16:44:30.245892 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:30.245832    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:44:30.339951 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:30.339918    2545 generic.go:356] "Generic (PLEG): container finished" podID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" containerID="f7e31393460de77afb2e0f933b93bcd64560008df764773d19f3bd9284faeac9" exitCode=0
Sep 11 16:44:30.340091 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:30.339993    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh" event={"ID":"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca","Type":"ContainerDied","Data":"f7e31393460de77afb2e0f933b93bcd64560008df764773d19f3bd9284faeac9"}
Sep 11 16:44:30.340348 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:30.340335    2545 scope.go:117] "RemoveContainer" containerID="f7e31393460de77afb2e0f933b93bcd64560008df764773d19f3bd9284faeac9"
Sep 11 16:44:30.341402 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:30.341377    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:44:30.341483 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:30.341467    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:44:30.341531 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:30.341520    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:44:31.347072 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:31.347037    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh" event={"ID":"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca","Type":"ContainerStarted","Data":"5eb2f78add27156f32b5e991883b8c67b815fce1593769358c6198f8802442c8"}
Sep 11 16:44:31.568926 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:31.568881    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:31.569160 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:31.569136    2545 patch_prober.go:28] interesting pod/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 11 16:44:31.569234 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:31.569186    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh" podUID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 11 16:44:32.569295 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:32.569266    2545 patch_prober.go:28] interesting pod/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 11 16:44:32.569653 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:32.569314    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh" podUID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 11 16:44:33.568661 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:33.568584    2545 patch_prober.go:28] interesting pod/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 11 16:44:33.568661 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:33.568645    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh" podUID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 11 16:44:34.569031 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:34.569000    2545 patch_prober.go:28] interesting pod/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 11 16:44:34.569423 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:34.569060    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh" podUID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 11 16:44:35.568933 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:35.568894    2545 patch_prober.go:28] interesting pod/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 11 16:44:35.569106 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:35.568953    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh" podUID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 11 16:44:36.572774 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:36.572744    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:36.573230 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:36.573077    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:36.573762 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:36.573741    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:37.292011 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:37.291980    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz"]
Sep 11 16:44:37.292394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:37.292380    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerName="istio-proxy"
Sep 11 16:44:37.292439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:37.292395    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerName="istio-proxy"
Sep 11 16:44:37.292473 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:37.292466    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerName="istio-proxy"
Sep 11 16:44:37.292510 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:37.292477    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerName="istio-proxy"
Sep 11 16:44:37.295735 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:37.295720    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz"
Sep 11 16:44:37.297939 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:37.297915    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:44:37.298053 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:37.298006    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:44:37.302891 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:37.302841    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz"]
Sep 11 16:44:37.418353 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:37.418328    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f96083c-894b-406e-a9b1-ec8f18b800dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-83625e23-9vssz\" (UID: \"0f96083c-894b-406e-a9b1-ec8f18b800dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz"
Sep 11 16:44:37.519832 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:37.519783    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f96083c-894b-406e-a9b1-ec8f18b800dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-83625e23-9vssz\" (UID: \"0f96083c-894b-406e-a9b1-ec8f18b800dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz"
Sep 11 16:44:37.520366 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:37.520341    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f96083c-894b-406e-a9b1-ec8f18b800dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-83625e23-9vssz\" (UID: \"0f96083c-894b-406e-a9b1-ec8f18b800dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz"
Sep 11 16:44:37.606791 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:37.606716    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz"
Sep 11 16:44:37.881541 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:37.881453    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz"]
Sep 11 16:44:37.883900 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:44:37.883856    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f96083c_894b_406e_a9b1_ec8f18b800dd.slice/crio-289c91e61d83ffd811eda84ac6e76b8727f064db3c5513c94cfd47beaf4272e0 WatchSource:0}: Error finding container 289c91e61d83ffd811eda84ac6e76b8727f064db3c5513c94cfd47beaf4272e0: Status 404 returned error can't find the container with id 289c91e61d83ffd811eda84ac6e76b8727f064db3c5513c94cfd47beaf4272e0
Sep 11 16:44:38.386122 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:38.386090    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz" event={"ID":"0f96083c-894b-406e-a9b1-ec8f18b800dd","Type":"ContainerStarted","Data":"fc638e23644714330fcb6901bf80707e31a04aacc9414a7b74aa0e3c58988ee7"}
Sep 11 16:44:38.386257 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:38.386143    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz" event={"ID":"0f96083c-894b-406e-a9b1-ec8f18b800dd","Type":"ContainerStarted","Data":"289c91e61d83ffd811eda84ac6e76b8727f064db3c5513c94cfd47beaf4272e0"}
Sep 11 16:44:38.401804 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:38.401748    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz" podStartSLOduration=1.401733657 podStartE2EDuration="1.401733657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:44:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:38.400079228 +0000 UTC m=+896.953249175" watchObservedRunningTime="2026-09-11 16:44:38.401733657 +0000 UTC m=+896.954903583"
Sep 11 16:44:39.393303 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:39.393263    2545 generic.go:356] "Generic (PLEG): container finished" podID="0f96083c-894b-406e-a9b1-ec8f18b800dd" containerID="fc638e23644714330fcb6901bf80707e31a04aacc9414a7b74aa0e3c58988ee7" exitCode=0
Sep 11 16:44:39.393753 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:39.393341    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz" event={"ID":"0f96083c-894b-406e-a9b1-ec8f18b800dd","Type":"ContainerDied","Data":"fc638e23644714330fcb6901bf80707e31a04aacc9414a7b74aa0e3c58988ee7"}
Sep 11 16:44:40.092025 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.091990    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"]
Sep 11 16:44:40.092645 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.092625    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerName="istio-proxy"
Sep 11 16:44:40.092716 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.092651    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c84535e-90d4-4066-88ac-67b11ccc89fd" containerName="istio-proxy"
Sep 11 16:44:40.096349 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.096326    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.098845 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.098825    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-0e8f65d9-kserve-self-signed-certs\""
Sep 11 16:44:40.107075 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.107051    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"]
Sep 11 16:44:40.245049 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.245025    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-model-cache\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.245173 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.245074    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsmkr\" (UniqueName: \"kubernetes.io/projected/6987ec5e-8aea-4862-94f4-4ede669cb052-kube-api-access-xsmkr\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.245173 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.245111    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-tmp-dir\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.245173 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.245130    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-home\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.245173 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.245143    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-dshm\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.245173 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.245156    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6987ec5e-8aea-4862-94f4-4ede669cb052-tls-certs\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.346103 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.346020    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-model-cache\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.346103 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.346085    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsmkr\" (UniqueName: \"kubernetes.io/projected/6987ec5e-8aea-4862-94f4-4ede669cb052-kube-api-access-xsmkr\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.346306 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.346117    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-tmp-dir\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.346306 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.346140    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-home\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.346306 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.346161    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-dshm\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.346306 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.346205    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6987ec5e-8aea-4862-94f4-4ede669cb052-tls-certs\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.346534 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.346500    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-model-cache\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.346534 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.346516    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-home\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.346687 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.346549    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-tmp-dir\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.348498 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.348477    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-dshm\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.348733 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.348716    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6987ec5e-8aea-4862-94f4-4ede669cb052-tls-certs\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.355393 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.355369    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsmkr\" (UniqueName: \"kubernetes.io/projected/6987ec5e-8aea-4862-94f4-4ede669cb052-kube-api-access-xsmkr\") pod \"e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") " pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.406154 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.406133    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:40.544247 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.544226    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz"
Sep 11 16:44:40.649227 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.649144    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f96083c-894b-406e-a9b1-ec8f18b800dd-maas-api-service-ca\") pod \"0f96083c-894b-406e-a9b1-ec8f18b800dd\" (UID: \"0f96083c-894b-406e-a9b1-ec8f18b800dd\") "
Sep 11 16:44:40.649473 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.649450    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f96083c-894b-406e-a9b1-ec8f18b800dd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0f96083c-894b-406e-a9b1-ec8f18b800dd" (UID: "0f96083c-894b-406e-a9b1-ec8f18b800dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:44:40.683365 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.683340    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"]
Sep 11 16:44:40.685081 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:44:40.685058    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6987ec5e_8aea_4862_94f4_4ede669cb052.slice/crio-b56a28634394a35c36dcd2d595247af1e9f6114d08dfc9dbf6902f8a283e4600 WatchSource:0}: Error finding container b56a28634394a35c36dcd2d595247af1e9f6114d08dfc9dbf6902f8a283e4600: Status 404 returned error can't find the container with id b56a28634394a35c36dcd2d595247af1e9f6114d08dfc9dbf6902f8a283e4600
Sep 11 16:44:40.750021 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:40.749998    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f96083c-894b-406e-a9b1-ec8f18b800dd-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:41.406440 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:41.406414    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz"
Sep 11 16:44:41.406894 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:41.406443    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz" event={"ID":"0f96083c-894b-406e-a9b1-ec8f18b800dd","Type":"ContainerDied","Data":"289c91e61d83ffd811eda84ac6e76b8727f064db3c5513c94cfd47beaf4272e0"}
Sep 11 16:44:41.406894 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:41.406493    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="289c91e61d83ffd811eda84ac6e76b8727f064db3c5513c94cfd47beaf4272e0"
Sep 11 16:44:41.408077 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:41.408053    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh" event={"ID":"6987ec5e-8aea-4862-94f4-4ede669cb052","Type":"ContainerStarted","Data":"cd727d1dce1bcdd654123d7fd5b9873603e9e32a3015e4cc65e7fcea38a5c3b6"}
Sep 11 16:44:41.408189 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:41.408084    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh" event={"ID":"6987ec5e-8aea-4862-94f4-4ede669cb052","Type":"ContainerStarted","Data":"b56a28634394a35c36dcd2d595247af1e9f6114d08dfc9dbf6902f8a283e4600"}
Sep 11 16:44:41.426610 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:41.426574    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh" podStartSLOduration=1.426561333 podStartE2EDuration="1.426561333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:44:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:44:41.425573049 +0000 UTC m=+899.978742990" watchObservedRunningTime="2026-09-11 16:44:41.426561333 +0000 UTC m=+899.979731313"
Sep 11 16:44:41.939453 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:41.939431    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:44:41.941265 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:41.941241    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:44:43.631405 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:43.631375    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz"]
Sep 11 16:44:43.635780 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:43.635756    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-83625e23-9vssz"]
Sep 11 16:44:43.938067 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:43.937991    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f96083c-894b-406e-a9b1-ec8f18b800dd" path="/var/lib/kubelet/pods/0f96083c-894b-406e-a9b1-ec8f18b800dd/volumes"
Sep 11 16:44:44.750459 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:44.750426    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"]
Sep 11 16:44:44.750820 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:44.750672    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh" podUID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" containerName="istio-proxy" containerID="cri-o://5eb2f78add27156f32b5e991883b8c67b815fce1593769358c6198f8802442c8" gracePeriod=30
Sep 11 16:44:44.795626 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:44.795595    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:44:50.010108 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.010075    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:50.036823 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.036747    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istiod-ca-cert\") pod \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") "
Sep 11 16:44:50.036992 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.036840    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7r9hl\" (UniqueName: \"kubernetes.io/projected/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-kube-api-access-7r9hl\") pod \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") "
Sep 11 16:44:50.036992 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.036893    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-credential-socket\") pod \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") "
Sep 11 16:44:50.036992 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.036922    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-podinfo\") pod \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") "
Sep 11 16:44:50.036992 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.036987    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-token\") pod \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") "
Sep 11 16:44:50.037204 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.037023    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-workload-certs\") pod \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") "
Sep 11 16:44:50.037204 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.037097    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-data\") pod \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") "
Sep 11 16:44:50.037204 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.037120    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" (UID: "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:44:50.037358 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.037219    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" (UID: "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:44:50.037358 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.037336    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" (UID: "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:44:50.037446 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.037409    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-envoy\") pod \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") "
Sep 11 16:44:50.037502 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.037451    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-data" (OuterVolumeSpecName: "istio-data") pod "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" (UID: "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:44:50.037502 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.037476    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-workload-socket\") pod \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\" (UID: \"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca\") "
Sep 11 16:44:50.037858 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.037842    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:50.037946 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.037894    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:50.037946 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.037893    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" (UID: "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:44:50.037946 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.037906    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:50.037946 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.037918    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:50.039348 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.039324    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-kube-api-access-7r9hl" (OuterVolumeSpecName: "kube-api-access-7r9hl") pod "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" (UID: "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca"). InnerVolumeSpecName "kube-api-access-7r9hl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:44:50.039473 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.039368    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-token" (OuterVolumeSpecName: "istio-token") pod "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" (UID: "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:44:50.039688 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.039671    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" (UID: "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:44:50.039954 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.039932    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" (UID: "eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:44:50.139235 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.139208    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:50.139235 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.139230    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:50.139235 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.139238    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:50.139428 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.139247    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:50.139428 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.139255    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7r9hl\" (UniqueName: \"kubernetes.io/projected/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca-kube-api-access-7r9hl\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:44:50.407065 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.406993    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:50.407065 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.407035    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:50.419147 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.419121    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:50.459241 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.459191    2545 generic.go:356] "Generic (PLEG): container finished" podID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" containerID="5eb2f78add27156f32b5e991883b8c67b815fce1593769358c6198f8802442c8" exitCode=0
Sep 11 16:44:50.459372 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.459242    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"
Sep 11 16:44:50.459372 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.459284    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh" event={"ID":"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca","Type":"ContainerDied","Data":"5eb2f78add27156f32b5e991883b8c67b815fce1593769358c6198f8802442c8"}
Sep 11 16:44:50.459372 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.459340    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh" event={"ID":"eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca","Type":"ContainerDied","Data":"b20adc8de95dd7270f4b4566c596a1c17bce062e418b169c3b50d5c9125dfe6d"}
Sep 11 16:44:50.459372 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.459368    2545 scope.go:117] "RemoveContainer" containerID="5eb2f78add27156f32b5e991883b8c67b815fce1593769358c6198f8802442c8"
Sep 11 16:44:50.469226 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.469209    2545 scope.go:117] "RemoveContainer" containerID="f7e31393460de77afb2e0f933b93bcd64560008df764773d19f3bd9284faeac9"
Sep 11 16:44:50.471791 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.471772    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:44:50.478296 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.478277    2545 scope.go:117] "RemoveContainer" containerID="5eb2f78add27156f32b5e991883b8c67b815fce1593769358c6198f8802442c8"
Sep 11 16:44:50.478552 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:44:50.478531    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eb2f78add27156f32b5e991883b8c67b815fce1593769358c6198f8802442c8\": container with ID starting with 5eb2f78add27156f32b5e991883b8c67b815fce1593769358c6198f8802442c8 not found: ID does not exist" containerID="5eb2f78add27156f32b5e991883b8c67b815fce1593769358c6198f8802442c8"
Sep 11 16:44:50.478613 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.478562    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eb2f78add27156f32b5e991883b8c67b815fce1593769358c6198f8802442c8"} err="failed to get container status \"5eb2f78add27156f32b5e991883b8c67b815fce1593769358c6198f8802442c8\": rpc error: code = NotFound desc = could not find container \"5eb2f78add27156f32b5e991883b8c67b815fce1593769358c6198f8802442c8\": container with ID starting with 5eb2f78add27156f32b5e991883b8c67b815fce1593769358c6198f8802442c8 not found: ID does not exist"
Sep 11 16:44:50.478613 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.478580    2545 scope.go:117] "RemoveContainer" containerID="f7e31393460de77afb2e0f933b93bcd64560008df764773d19f3bd9284faeac9"
Sep 11 16:44:50.478953 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:44:50.478921    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7e31393460de77afb2e0f933b93bcd64560008df764773d19f3bd9284faeac9\": container with ID starting with f7e31393460de77afb2e0f933b93bcd64560008df764773d19f3bd9284faeac9 not found: ID does not exist" containerID="f7e31393460de77afb2e0f933b93bcd64560008df764773d19f3bd9284faeac9"
Sep 11 16:44:50.479040 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.478956    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7e31393460de77afb2e0f933b93bcd64560008df764773d19f3bd9284faeac9"} err="failed to get container status \"f7e31393460de77afb2e0f933b93bcd64560008df764773d19f3bd9284faeac9\": rpc error: code = NotFound desc = could not find container \"f7e31393460de77afb2e0f933b93bcd64560008df764773d19f3bd9284faeac9\": container with ID starting with f7e31393460de77afb2e0f933b93bcd64560008df764773d19f3bd9284faeac9 not found: ID does not exist"
Sep 11 16:44:50.491362 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.491338    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"]
Sep 11 16:44:50.498584 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:50.498564    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-83625e23-openshift-default-79ccb8c56-mgqmh"]
Sep 11 16:44:51.941884 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:51.939033    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" path="/var/lib/kubelet/pods/eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca/volumes"
Sep 11 16:44:56.551059 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:44:56.551029    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:45:00.211344 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.211310    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-4bf042d5-29819085-6gz9m"]
Sep 11 16:45:00.211698 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.211690    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" containerName="istio-proxy"
Sep 11 16:45:00.211746 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.211701    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" containerName="istio-proxy"
Sep 11 16:45:00.211746 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.211716    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f96083c-894b-406e-a9b1-ec8f18b800dd" containerName="revoke-keys"
Sep 11 16:45:00.211746 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.211722    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f96083c-894b-406e-a9b1-ec8f18b800dd" containerName="revoke-keys"
Sep 11 16:45:00.211892 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.211791    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" containerName="istio-proxy"
Sep 11 16:45:00.211892 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.211803    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f96083c-894b-406e-a9b1-ec8f18b800dd" containerName="revoke-keys"
Sep 11 16:45:00.216518 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.216498    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819085-pdbhh"]
Sep 11 16:45:00.216621 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.216608    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-4bf042d5-29819085-6gz9m"
Sep 11 16:45:00.216846 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.216832    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" containerName="istio-proxy"
Sep 11 16:45:00.216950 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.216848    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" containerName="istio-proxy"
Sep 11 16:45:00.217006 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.216962    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb6b7741-2fd4-48da-adc6-0cd3b0ff46ca" containerName="istio-proxy"
Sep 11 16:45:00.219845 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.219828    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819085-pdbhh"
Sep 11 16:45:00.224432 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.224410    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:45:00.227033 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.227015    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-4bf042d5-29819085-6gz9m"
Sep 11 16:45:00.233884 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.233851    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819085-pdbhh"
Sep 11 16:45:00.235679 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.235646    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4264bf-29819085-dvk5p"]
Sep 11 16:45:00.241539 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.240775    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-42752c-29819085-zrvtw"]
Sep 11 16:45:00.241657 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.241637    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4264bf-29819085-dvk5p"
Sep 11 16:45:00.244787 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.244765    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-4bf042d5-29819085-6gz9m"]
Sep 11 16:45:00.244924 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.244885    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-42752c-29819085-zrvtw"
Sep 11 16:45:00.253396 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.253366    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4264bf-29819085-dvk5p"]
Sep 11 16:45:00.258617 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.258549    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-42752c-29819085-zrvtw"]
Sep 11 16:45:00.261732 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.261583    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819085-pdbhh"]
Sep 11 16:45:00.283311 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.283093    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4264bf-29819085-dvk5p"
Sep 11 16:45:00.292725 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.292672    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-42752c-29819085-zrvtw"
Sep 11 16:45:00.531189 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.531163    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-4bf042d5-29819085-6gz9m"]
Sep 11 16:45:00.532637 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:45:00.532610    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7a759d_5300_4f21_a793_9283786e12eb.slice/crio-3f9653cbe451c7eac37e31f821e8f4739d7eaebc8048b088a6d59c827aeb8228 WatchSource:0}: Error finding container 3f9653cbe451c7eac37e31f821e8f4739d7eaebc8048b088a6d59c827aeb8228: Status 404 returned error can't find the container with id 3f9653cbe451c7eac37e31f821e8f4739d7eaebc8048b088a6d59c827aeb8228
Sep 11 16:45:00.539557 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.539501    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819085-pdbhh"]
Sep 11 16:45:00.541423 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:45:00.541396    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79fd3e82_3bde_4971_a3c4_2af91531d3d4.slice/crio-e90bd4701e0987a20b6918b9d517e99e10801dc940ed83b50ec5ac4627ebd9fa WatchSource:0}: Error finding container e90bd4701e0987a20b6918b9d517e99e10801dc940ed83b50ec5ac4627ebd9fa: Status 404 returned error can't find the container with id e90bd4701e0987a20b6918b9d517e99e10801dc940ed83b50ec5ac4627ebd9fa
Sep 11 16:45:00.593923 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.593901    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4264bf-29819085-dvk5p"]
Sep 11 16:45:00.594635 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:45:00.594610    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a5a538_6ecc_4ed7_b381_479bac30bea4.slice/crio-0d2675c3af1cf87d7556cbb68ebaefad7d22571c3bd823c49541e01ba689adaa WatchSource:0}: Error finding container 0d2675c3af1cf87d7556cbb68ebaefad7d22571c3bd823c49541e01ba689adaa: Status 404 returned error can't find the container with id 0d2675c3af1cf87d7556cbb68ebaefad7d22571c3bd823c49541e01ba689adaa
Sep 11 16:45:00.598895 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:00.598859    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-42752c-29819085-zrvtw"]
Sep 11 16:45:00.601655 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:45:00.601624    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45fe1529_f065_4929_8037_8cb699c077c7.slice/crio-0ad2920e65322f1f80b2f25ca5a9002a4fb24895b248883e41209fa78f460fb4 WatchSource:0}: Error finding container 0ad2920e65322f1f80b2f25ca5a9002a4fb24895b248883e41209fa78f460fb4: Status 404 returned error can't find the container with id 0ad2920e65322f1f80b2f25ca5a9002a4fb24895b248883e41209fa78f460fb4
Sep 11 16:45:01.520841 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:01.520804    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4264bf-29819085-dvk5p" event={"ID":"73a5a538-6ecc-4ed7-b381-479bac30bea4","Type":"ContainerStarted","Data":"a747f976bb3aba0c7a5c2877fb820122f866ff2f5ce3b6020200cd194a9720b1"}
Sep 11 16:45:01.521205 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:01.520877    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4264bf-29819085-dvk5p" event={"ID":"73a5a538-6ecc-4ed7-b381-479bac30bea4","Type":"ContainerStarted","Data":"0d2675c3af1cf87d7556cbb68ebaefad7d22571c3bd823c49541e01ba689adaa"}
Sep 11 16:45:01.522521 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:01.522494    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819085-pdbhh" event={"ID":"79fd3e82-3bde-4971-a3c4-2af91531d3d4","Type":"ContainerStarted","Data":"fa9ad13b6f65cf2a29577316b51d3dff0cd4c2bc2c96649dcc94ff25bf63ddcc"}
Sep 11 16:45:01.522618 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:01.522540    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819085-pdbhh" event={"ID":"79fd3e82-3bde-4971-a3c4-2af91531d3d4","Type":"ContainerStarted","Data":"e90bd4701e0987a20b6918b9d517e99e10801dc940ed83b50ec5ac4627ebd9fa"}
Sep 11 16:45:01.524035 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:01.524009    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-42752c-29819085-zrvtw" event={"ID":"45fe1529-f065-4929-8037-8cb699c077c7","Type":"ContainerStarted","Data":"e8f93196328682d272330465040da7a508aa861643555e53f25bc8c3e23a8db9"}
Sep 11 16:45:01.524155 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:01.524060    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-42752c-29819085-zrvtw" event={"ID":"45fe1529-f065-4929-8037-8cb699c077c7","Type":"ContainerStarted","Data":"0ad2920e65322f1f80b2f25ca5a9002a4fb24895b248883e41209fa78f460fb4"}
Sep 11 16:45:01.525446 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:01.525425    2545 generic.go:356] "Generic (PLEG): container finished" podID="5b7a759d-5300-4f21-a793-9283786e12eb" containerID="3e9b6e4733b01f270e9fd6ebe96b4b105eb513c61a8ebcef8bdf8578d7ced593" exitCode=0
Sep 11 16:45:01.525526 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:01.525458    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-4bf042d5-29819085-6gz9m" event={"ID":"5b7a759d-5300-4f21-a793-9283786e12eb","Type":"ContainerDied","Data":"3e9b6e4733b01f270e9fd6ebe96b4b105eb513c61a8ebcef8bdf8578d7ced593"}
Sep 11 16:45:01.525526 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:01.525485    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-4bf042d5-29819085-6gz9m" event={"ID":"5b7a759d-5300-4f21-a793-9283786e12eb","Type":"ContainerStarted","Data":"3f9653cbe451c7eac37e31f821e8f4739d7eaebc8048b088a6d59c827aeb8228"}
Sep 11 16:45:01.549702 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:01.549660    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4264bf-29819085-dvk5p" podStartSLOduration=1.549646905 podStartE2EDuration="1.549646905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:45:01.547986322 +0000 UTC m=+920.101156269" watchObservedRunningTime="2026-09-11 16:45:01.549646905 +0000 UTC m=+920.102816845"
Sep 11 16:45:01.573774 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:01.573735    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819085-pdbhh" podStartSLOduration=1.573722987 podStartE2EDuration="1.573722987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:45:01.573164003 +0000 UTC m=+920.126333952" watchObservedRunningTime="2026-09-11 16:45:01.573722987 +0000 UTC m=+920.126892907"
Sep 11 16:45:01.612448 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:01.612401    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-42752c-29819085-zrvtw" podStartSLOduration=1.612380986 podStartE2EDuration="1.612380986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:45:01.612342039 +0000 UTC m=+920.165511982" watchObservedRunningTime="2026-09-11 16:45:01.612380986 +0000 UTC m=+920.165550916"
Sep 11 16:45:02.532235 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:02.532203    2545 generic.go:356] "Generic (PLEG): container finished" podID="73a5a538-6ecc-4ed7-b381-479bac30bea4" containerID="a747f976bb3aba0c7a5c2877fb820122f866ff2f5ce3b6020200cd194a9720b1" exitCode=0
Sep 11 16:45:02.532607 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:02.532291    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4264bf-29819085-dvk5p" event={"ID":"73a5a538-6ecc-4ed7-b381-479bac30bea4","Type":"ContainerDied","Data":"a747f976bb3aba0c7a5c2877fb820122f866ff2f5ce3b6020200cd194a9720b1"}
Sep 11 16:45:02.533747 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:02.533722    2545 generic.go:356] "Generic (PLEG): container finished" podID="79fd3e82-3bde-4971-a3c4-2af91531d3d4" containerID="fa9ad13b6f65cf2a29577316b51d3dff0cd4c2bc2c96649dcc94ff25bf63ddcc" exitCode=0
Sep 11 16:45:02.533902 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:02.533810    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819085-pdbhh" event={"ID":"79fd3e82-3bde-4971-a3c4-2af91531d3d4","Type":"ContainerDied","Data":"fa9ad13b6f65cf2a29577316b51d3dff0cd4c2bc2c96649dcc94ff25bf63ddcc"}
Sep 11 16:45:02.535279 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:02.535258    2545 generic.go:356] "Generic (PLEG): container finished" podID="45fe1529-f065-4929-8037-8cb699c077c7" containerID="e8f93196328682d272330465040da7a508aa861643555e53f25bc8c3e23a8db9" exitCode=0
Sep 11 16:45:02.535363 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:02.535320    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-42752c-29819085-zrvtw" event={"ID":"45fe1529-f065-4929-8037-8cb699c077c7","Type":"ContainerDied","Data":"e8f93196328682d272330465040da7a508aa861643555e53f25bc8c3e23a8db9"}
Sep 11 16:45:02.675616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:02.675589    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-4bf042d5-29819085-6gz9m"
Sep 11 16:45:03.543094 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:03.543061    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-4bf042d5-29819085-6gz9m"
Sep 11 16:45:03.543094 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:03.543070    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-4bf042d5-29819085-6gz9m" event={"ID":"5b7a759d-5300-4f21-a793-9283786e12eb","Type":"ContainerDied","Data":"3f9653cbe451c7eac37e31f821e8f4739d7eaebc8048b088a6d59c827aeb8228"}
Sep 11 16:45:03.543602 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:03.543126    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f9653cbe451c7eac37e31f821e8f4739d7eaebc8048b088a6d59c827aeb8228"
Sep 11 16:45:03.686230 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:03.686205    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819085-pdbhh"
Sep 11 16:45:03.734118 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:03.734098    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4264bf-29819085-dvk5p"
Sep 11 16:45:03.737044 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:03.737025    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-42752c-29819085-zrvtw"
Sep 11 16:45:04.549556 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:04.549529    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4264bf-29819085-dvk5p"
Sep 11 16:45:04.549556 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:04.549540    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4264bf-29819085-dvk5p" event={"ID":"73a5a538-6ecc-4ed7-b381-479bac30bea4","Type":"ContainerDied","Data":"0d2675c3af1cf87d7556cbb68ebaefad7d22571c3bd823c49541e01ba689adaa"}
Sep 11 16:45:04.550047 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:04.549589    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d2675c3af1cf87d7556cbb68ebaefad7d22571c3bd823c49541e01ba689adaa"
Sep 11 16:45:04.550936 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:04.550913    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819085-pdbhh"
Sep 11 16:45:04.550936 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:04.550933    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819085-pdbhh" event={"ID":"79fd3e82-3bde-4971-a3c4-2af91531d3d4","Type":"ContainerDied","Data":"e90bd4701e0987a20b6918b9d517e99e10801dc940ed83b50ec5ac4627ebd9fa"}
Sep 11 16:45:04.551133 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:04.550965    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e90bd4701e0987a20b6918b9d517e99e10801dc940ed83b50ec5ac4627ebd9fa"
Sep 11 16:45:04.552372 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:04.552339    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-42752c-29819085-zrvtw"
Sep 11 16:45:04.552500 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:04.552342    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-42752c-29819085-zrvtw" event={"ID":"45fe1529-f065-4929-8037-8cb699c077c7","Type":"ContainerDied","Data":"0ad2920e65322f1f80b2f25ca5a9002a4fb24895b248883e41209fa78f460fb4"}
Sep 11 16:45:04.552568 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:04.552507    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ad2920e65322f1f80b2f25ca5a9002a4fb24895b248883e41209fa78f460fb4"
Sep 11 16:45:07.629859 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.629830    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"]
Sep 11 16:45:07.630291 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.630277    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a5a538-6ecc-4ed7-b381-479bac30bea4" containerName="cleanup"
Sep 11 16:45:07.630333 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.630293    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a5a538-6ecc-4ed7-b381-479bac30bea4" containerName="cleanup"
Sep 11 16:45:07.630333 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.630305    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45fe1529-f065-4929-8037-8cb699c077c7" containerName="cleanup"
Sep 11 16:45:07.630333 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.630310    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="45fe1529-f065-4929-8037-8cb699c077c7" containerName="cleanup"
Sep 11 16:45:07.630333 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.630321    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79fd3e82-3bde-4971-a3c4-2af91531d3d4" containerName="cleanup"
Sep 11 16:45:07.630333 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.630326    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="79fd3e82-3bde-4971-a3c4-2af91531d3d4" containerName="cleanup"
Sep 11 16:45:07.630489 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.630335    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b7a759d-5300-4f21-a793-9283786e12eb" containerName="cleanup"
Sep 11 16:45:07.630489 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.630343    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b7a759d-5300-4f21-a793-9283786e12eb" containerName="cleanup"
Sep 11 16:45:07.630489 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.630412    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="73a5a538-6ecc-4ed7-b381-479bac30bea4" containerName="cleanup"
Sep 11 16:45:07.630489 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.630425    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="79fd3e82-3bde-4971-a3c4-2af91531d3d4" containerName="cleanup"
Sep 11 16:45:07.630489 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.630436    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="45fe1529-f065-4929-8037-8cb699c077c7" containerName="cleanup"
Sep 11 16:45:07.630489 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.630443    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b7a759d-5300-4f21-a793-9283786e12eb" containerName="cleanup"
Sep 11 16:45:07.633482 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.633460    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.636486 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.636463    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-42752c\"/\"openshift-service-ca.crt\""
Sep 11 16:45:07.636486 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.636479    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-42752c\"/\"kube-root-ca.crt\""
Sep 11 16:45:07.636665 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.636480    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-42752c\"/\"autores-model-42752c-kserve-self-signed-certs\""
Sep 11 16:45:07.636665 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.636624    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-42752c\"/\"default-dockercfg-d6zfq\""
Sep 11 16:45:07.647138 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.647114    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"]
Sep 11 16:45:07.700074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.700045    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-dshm\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.700074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.700075    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-tls-certs\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.700261 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.700093    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-tmp-dir\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.700261 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.700111    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrl45\" (UniqueName: \"kubernetes.io/projected/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-kube-api-access-xrl45\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.700261 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.700235    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-home\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.700378 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.700268    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-model-cache\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.801190 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.801157    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-dshm\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.801190 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.801193    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-tls-certs\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.801426 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.801210    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-tmp-dir\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.801426 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.801234    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrl45\" (UniqueName: \"kubernetes.io/projected/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-kube-api-access-xrl45\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.801426 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.801315    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-home\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.801426 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.801346    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-model-cache\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.801626 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.801610    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-tmp-dir\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.801729 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.801695    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-home\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.801856 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.801748    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-model-cache\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.803654 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.803637    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-dshm\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.803808 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.803793    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-tls-certs\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.810406 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.810387    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrl45\" (UniqueName: \"kubernetes.io/projected/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-kube-api-access-xrl45\") pod \"autores-model-42752c-kserve-657d5c499f-qxtpn\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") " pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:07.943328 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:07.943245    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:08.216114 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:08.216092    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"]
Sep 11 16:45:08.218360 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:45:08.218329    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf11d7713_5c6c_4d67_a79b_0e82c98a07e0.slice/crio-f9465647e0ddc1365a3ff1e161afaf8f29778cf3d9fdc403d0f55f3e84e3518b WatchSource:0}: Error finding container f9465647e0ddc1365a3ff1e161afaf8f29778cf3d9fdc403d0f55f3e84e3518b: Status 404 returned error can't find the container with id f9465647e0ddc1365a3ff1e161afaf8f29778cf3d9fdc403d0f55f3e84e3518b
Sep 11 16:45:08.579248 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:08.579213    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn" event={"ID":"f11d7713-5c6c-4d67-a79b-0e82c98a07e0","Type":"ContainerStarted","Data":"85c8d0b21d812bec1bd695fd66b8743872985e274288237723e22ec36fe93a9e"}
Sep 11 16:45:08.579248 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:08.579260    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn" event={"ID":"f11d7713-5c6c-4d67-a79b-0e82c98a07e0","Type":"ContainerStarted","Data":"f9465647e0ddc1365a3ff1e161afaf8f29778cf3d9fdc403d0f55f3e84e3518b"}
Sep 11 16:45:08.601635 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:08.601586    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn" podStartSLOduration=1.601569201 podStartE2EDuration="1.601569201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:45:08.59998376 +0000 UTC m=+927.153153686" watchObservedRunningTime="2026-09-11 16:45:08.601569201 +0000 UTC m=+927.154739126"
Sep 11 16:45:12.708577 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:12.708542    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf"]
Sep 11 16:45:12.711903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:12.711887    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf"
Sep 11 16:45:12.714197 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:12.714178    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:45:12.714335 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:12.714282    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:45:12.719859 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:12.719812    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf"]
Sep 11 16:45:12.847879 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:12.847817    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db60e8f7-764d-4e70-b210-470f82ad428f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf\" (UID: \"db60e8f7-764d-4e70-b210-470f82ad428f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf"
Sep 11 16:45:12.948690 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:12.948660    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db60e8f7-764d-4e70-b210-470f82ad428f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf\" (UID: \"db60e8f7-764d-4e70-b210-470f82ad428f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf"
Sep 11 16:45:12.949266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:12.949247    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db60e8f7-764d-4e70-b210-470f82ad428f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf\" (UID: \"db60e8f7-764d-4e70-b210-470f82ad428f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf"
Sep 11 16:45:13.024042 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:13.024016    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf"
Sep 11 16:45:13.179016 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:13.178982    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:45:13.310835 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:13.310810    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf"]
Sep 11 16:45:13.311258 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:45:13.311234    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb60e8f7_764d_4e70_b210_470f82ad428f.slice/crio-ec283ae3e314f29573bfbb99dd99d42819ba6f3169c64cb042936787c64f27fc WatchSource:0}: Error finding container ec283ae3e314f29573bfbb99dd99d42819ba6f3169c64cb042936787c64f27fc: Status 404 returned error can't find the container with id ec283ae3e314f29573bfbb99dd99d42819ba6f3169c64cb042936787c64f27fc
Sep 11 16:45:13.609723 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:13.609692    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf" event={"ID":"db60e8f7-764d-4e70-b210-470f82ad428f","Type":"ContainerStarted","Data":"fc624d7509ea062ab2b8ea46cc791dee28d638850b057619a187cc8b2eed488b"}
Sep 11 16:45:13.609831 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:13.609749    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf" event={"ID":"db60e8f7-764d-4e70-b210-470f82ad428f","Type":"ContainerStarted","Data":"ec283ae3e314f29573bfbb99dd99d42819ba6f3169c64cb042936787c64f27fc"}
Sep 11 16:45:13.630966 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:13.630928    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf" podStartSLOduration=1.6309135270000001 podStartE2EDuration="1.630913527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:45:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:45:13.629837177 +0000 UTC m=+932.183007103" watchObservedRunningTime="2026-09-11 16:45:13.630913527 +0000 UTC m=+932.184083453"
Sep 11 16:45:14.617437 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:14.617401    2545 generic.go:356] "Generic (PLEG): container finished" podID="db60e8f7-764d-4e70-b210-470f82ad428f" containerID="fc624d7509ea062ab2b8ea46cc791dee28d638850b057619a187cc8b2eed488b" exitCode=0
Sep 11 16:45:14.617827 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:14.617495    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf" event={"ID":"db60e8f7-764d-4e70-b210-470f82ad428f","Type":"ContainerDied","Data":"fc624d7509ea062ab2b8ea46cc791dee28d638850b057619a187cc8b2eed488b"}
Sep 11 16:45:15.755759 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:15.755736    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf"
Sep 11 16:45:15.879589 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:15.879511    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db60e8f7-764d-4e70-b210-470f82ad428f-maas-api-service-ca\") pod \"db60e8f7-764d-4e70-b210-470f82ad428f\" (UID: \"db60e8f7-764d-4e70-b210-470f82ad428f\") "
Sep 11 16:45:15.879957 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:15.879928    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db60e8f7-764d-4e70-b210-470f82ad428f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "db60e8f7-764d-4e70-b210-470f82ad428f" (UID: "db60e8f7-764d-4e70-b210-470f82ad428f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:15.980330 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:15.980303    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db60e8f7-764d-4e70-b210-470f82ad428f-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:16.629392 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:16.629369    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf"
Sep 11 16:45:16.629562 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:16.629362    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf" event={"ID":"db60e8f7-764d-4e70-b210-470f82ad428f","Type":"ContainerDied","Data":"ec283ae3e314f29573bfbb99dd99d42819ba6f3169c64cb042936787c64f27fc"}
Sep 11 16:45:16.629562 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:16.629498    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec283ae3e314f29573bfbb99dd99d42819ba6f3169c64cb042936787c64f27fc"
Sep 11 16:45:17.944127 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:17.944101    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:17.944481 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:17.944138    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:17.956537 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:17.956517    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:18.650951 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:18.650923    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:19.842947 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:19.842904    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-4bf042d5-29819085-6gz9m"]
Sep 11 16:45:19.846492 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:19.846467    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-4bf042d5-29819085-6gz9m"]
Sep 11 16:45:19.945079 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:19.945046    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b7a759d-5300-4f21-a793-9283786e12eb" path="/var/lib/kubelet/pods/5b7a759d-5300-4f21-a793-9283786e12eb/volumes"
Sep 11 16:45:20.220001 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:20.219856    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf"]
Sep 11 16:45:20.225248 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:20.225218    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4bf042d5-w5zxf"]
Sep 11 16:45:21.938182 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:21.938147    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db60e8f7-764d-4e70-b210-470f82ad428f" path="/var/lib/kubelet/pods/db60e8f7-764d-4e70-b210-470f82ad428f/volumes"
Sep 11 16:45:26.845542 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:26.845511    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:45:29.894092 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:29.894057    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"]
Sep 11 16:45:30.420769 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:30.420739    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"]
Sep 11 16:45:31.929045 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:31.929016    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"]
Sep 11 16:45:32.056846 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:32.056817    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp"]
Sep 11 16:45:32.057281 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:32.057267    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db60e8f7-764d-4e70-b210-470f82ad428f" containerName="revoke-keys"
Sep 11 16:45:32.057327 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:32.057283    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="db60e8f7-764d-4e70-b210-470f82ad428f" containerName="revoke-keys"
Sep 11 16:45:32.057385 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:32.057374    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="db60e8f7-764d-4e70-b210-470f82ad428f" containerName="revoke-keys"
Sep 11 16:45:32.061992 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:32.061975    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp"
Sep 11 16:45:32.064276 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:32.064144    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:45:32.064276 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:32.064205    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:45:32.075057 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:32.075031    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp"]
Sep 11 16:45:32.133092 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:32.133060    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cf5cc26-3af8-4a30-aca4-07c3faadb98b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-42752c-xr4mp\" (UID: \"2cf5cc26-3af8-4a30-aca4-07c3faadb98b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp"
Sep 11 16:45:32.233811 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:32.233727    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cf5cc26-3af8-4a30-aca4-07c3faadb98b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-42752c-xr4mp\" (UID: \"2cf5cc26-3af8-4a30-aca4-07c3faadb98b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp"
Sep 11 16:45:32.234347 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:32.234330    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cf5cc26-3af8-4a30-aca4-07c3faadb98b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-42752c-xr4mp\" (UID: \"2cf5cc26-3af8-4a30-aca4-07c3faadb98b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp"
Sep 11 16:45:32.382311 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:32.382280    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp"
Sep 11 16:45:32.665903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:32.665833    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp"]
Sep 11 16:45:32.666438 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:45:32.666416    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cf5cc26_3af8_4a30_aca4_07c3faadb98b.slice/crio-41b343d3b8d20136b5d20240df3c47eaa1020ca04804e4d66fd764b3624128fc WatchSource:0}: Error finding container 41b343d3b8d20136b5d20240df3c47eaa1020ca04804e4d66fd764b3624128fc: Status 404 returned error can't find the container with id 41b343d3b8d20136b5d20240df3c47eaa1020ca04804e4d66fd764b3624128fc
Sep 11 16:45:32.714836 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:32.714806    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp" event={"ID":"2cf5cc26-3af8-4a30-aca4-07c3faadb98b","Type":"ContainerStarted","Data":"41b343d3b8d20136b5d20240df3c47eaa1020ca04804e4d66fd764b3624128fc"}
Sep 11 16:45:33.722666 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:33.722579    2545 generic.go:356] "Generic (PLEG): container finished" podID="2cf5cc26-3af8-4a30-aca4-07c3faadb98b" containerID="d43ddee46264aea3935e4d1126eca43a68fc0b4227b12218f96f5fcda5c80cf9" exitCode=0
Sep 11 16:45:33.723047 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:33.722672    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp" event={"ID":"2cf5cc26-3af8-4a30-aca4-07c3faadb98b","Type":"ContainerDied","Data":"d43ddee46264aea3935e4d1126eca43a68fc0b4227b12218f96f5fcda5c80cf9"}
Sep 11 16:45:34.865750 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:34.865729    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp"
Sep 11 16:45:34.961400 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:34.961368    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cf5cc26-3af8-4a30-aca4-07c3faadb98b-maas-api-service-ca\") pod \"2cf5cc26-3af8-4a30-aca4-07c3faadb98b\" (UID: \"2cf5cc26-3af8-4a30-aca4-07c3faadb98b\") "
Sep 11 16:45:34.961689 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:34.961668    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cf5cc26-3af8-4a30-aca4-07c3faadb98b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2cf5cc26-3af8-4a30-aca4-07c3faadb98b" (UID: "2cf5cc26-3af8-4a30-aca4-07c3faadb98b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:35.062505 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:35.062472    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cf5cc26-3af8-4a30-aca4-07c3faadb98b-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:35.735483 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:35.735455    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp"
Sep 11 16:45:35.735483 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:35.735460    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp" event={"ID":"2cf5cc26-3af8-4a30-aca4-07c3faadb98b","Type":"ContainerDied","Data":"41b343d3b8d20136b5d20240df3c47eaa1020ca04804e4d66fd764b3624128fc"}
Sep 11 16:45:35.735676 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:35.735502    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41b343d3b8d20136b5d20240df3c47eaa1020ca04804e4d66fd764b3624128fc"
Sep 11 16:45:41.645097 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:41.645069    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:45:42.896417 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:42.896380    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-42752c-29819085-zrvtw"]
Sep 11 16:45:42.900480 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:42.900455    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-42752c-29819085-zrvtw"]
Sep 11 16:45:42.934326 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:42.934293    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"]
Sep 11 16:45:42.934847 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:42.934789    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs" podUID="6530b7d2-6dc7-4792-a774-61673f74d10e" containerName="payload-pre-processing" containerID="cri-o://0ac02a5466cfff20410b5a622c2e357776c46568df4b570df2d9aa4a336057c4" gracePeriod=30
Sep 11 16:45:43.200289 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.200265    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"
Sep 11 16:45:43.279948 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.279916    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp"]
Sep 11 16:45:43.282910 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.282886    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-42752c-xr4mp"]
Sep 11 16:45:43.338329 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.338303    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w88s\" (UniqueName: \"kubernetes.io/projected/6530b7d2-6dc7-4792-a774-61673f74d10e-kube-api-access-7w88s\") pod \"6530b7d2-6dc7-4792-a774-61673f74d10e\" (UID: \"6530b7d2-6dc7-4792-a774-61673f74d10e\") "
Sep 11 16:45:43.338482 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.338380    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6530b7d2-6dc7-4792-a774-61673f74d10e-plugins-config-volume\") pod \"6530b7d2-6dc7-4792-a774-61673f74d10e\" (UID: \"6530b7d2-6dc7-4792-a774-61673f74d10e\") "
Sep 11 16:45:43.338680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.338661    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6530b7d2-6dc7-4792-a774-61673f74d10e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6530b7d2-6dc7-4792-a774-61673f74d10e" (UID: "6530b7d2-6dc7-4792-a774-61673f74d10e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:43.338825 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.338804    2545 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6530b7d2-6dc7-4792-a774-61673f74d10e-plugins-config-volume\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:43.340453 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.340436    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6530b7d2-6dc7-4792-a774-61673f74d10e-kube-api-access-7w88s" (OuterVolumeSpecName: "kube-api-access-7w88s") pod "6530b7d2-6dc7-4792-a774-61673f74d10e" (UID: "6530b7d2-6dc7-4792-a774-61673f74d10e"). InnerVolumeSpecName "kube-api-access-7w88s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:45:43.439319 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.439261    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w88s\" (UniqueName: \"kubernetes.io/projected/6530b7d2-6dc7-4792-a774-61673f74d10e-kube-api-access-7w88s\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:43.783617 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.783584    2545 generic.go:356] "Generic (PLEG): container finished" podID="6530b7d2-6dc7-4792-a774-61673f74d10e" containerID="0ac02a5466cfff20410b5a622c2e357776c46568df4b570df2d9aa4a336057c4" exitCode=0
Sep 11 16:45:43.783930 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.783629    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"
Sep 11 16:45:43.783930 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.783643    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs" event={"ID":"6530b7d2-6dc7-4792-a774-61673f74d10e","Type":"ContainerDied","Data":"0ac02a5466cfff20410b5a622c2e357776c46568df4b570df2d9aa4a336057c4"}
Sep 11 16:45:43.783930 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.783685    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs" event={"ID":"6530b7d2-6dc7-4792-a774-61673f74d10e","Type":"ContainerDied","Data":"0c3d74addc9affac9a6321d34b256c0eaff3900035be1bb1bbd5cb343ce7d9af"}
Sep 11 16:45:43.783930 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.783706    2545 scope.go:117] "RemoveContainer" containerID="0ac02a5466cfff20410b5a622c2e357776c46568df4b570df2d9aa4a336057c4"
Sep 11 16:45:43.794111 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.794091    2545 scope.go:117] "RemoveContainer" containerID="0ac02a5466cfff20410b5a622c2e357776c46568df4b570df2d9aa4a336057c4"
Sep 11 16:45:43.794398 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:45:43.794378    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ac02a5466cfff20410b5a622c2e357776c46568df4b570df2d9aa4a336057c4\": container with ID starting with 0ac02a5466cfff20410b5a622c2e357776c46568df4b570df2d9aa4a336057c4 not found: ID does not exist" containerID="0ac02a5466cfff20410b5a622c2e357776c46568df4b570df2d9aa4a336057c4"
Sep 11 16:45:43.794469 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.794411    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ac02a5466cfff20410b5a622c2e357776c46568df4b570df2d9aa4a336057c4"} err="failed to get container status \"0ac02a5466cfff20410b5a622c2e357776c46568df4b570df2d9aa4a336057c4\": rpc error: code = NotFound desc = could not find container \"0ac02a5466cfff20410b5a622c2e357776c46568df4b570df2d9aa4a336057c4\": container with ID starting with 0ac02a5466cfff20410b5a622c2e357776c46568df4b570df2d9aa4a336057c4 not found: ID does not exist"
Sep 11 16:45:43.817230 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.817211    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"]
Sep 11 16:45:43.820464 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.820440    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-42752c-cc76cd8db-z8mbs"]
Sep 11 16:45:43.937884 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.937831    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cf5cc26-3af8-4a30-aca4-07c3faadb98b" path="/var/lib/kubelet/pods/2cf5cc26-3af8-4a30-aca4-07c3faadb98b/volumes"
Sep 11 16:45:43.938217 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.938143    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45fe1529-f065-4929-8037-8cb699c077c7" path="/var/lib/kubelet/pods/45fe1529-f065-4929-8037-8cb699c077c7/volumes"
Sep 11 16:45:43.938413 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:43.938400    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6530b7d2-6dc7-4792-a774-61673f74d10e" path="/var/lib/kubelet/pods/6530b7d2-6dc7-4792-a774-61673f74d10e/volumes"
Sep 11 16:45:44.913013 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:44.912952    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s" podUID="a984c444-0603-4838-acc1-b99ef37c6706" containerName="main" containerID="cri-o://90c4b704dc8e9485c9d694d06b71ada9ec7d372d10f615a8036e3b6b87250a3e" gracePeriod=45
Sep 11 16:45:45.155991 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.155960    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:45:45.254566 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.254538    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a984c444-0603-4838-acc1-b99ef37c6706-tls-certs\") pod \"a984c444-0603-4838-acc1-b99ef37c6706\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") "
Sep 11 16:45:45.254738 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.254573    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-model-cache\") pod \"a984c444-0603-4838-acc1-b99ef37c6706\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") "
Sep 11 16:45:45.254738 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.254635    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-dshm\") pod \"a984c444-0603-4838-acc1-b99ef37c6706\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") "
Sep 11 16:45:45.254738 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.254676    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-tmp-dir\") pod \"a984c444-0603-4838-acc1-b99ef37c6706\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") "
Sep 11 16:45:45.254738 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.254723    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzv7n\" (UniqueName: \"kubernetes.io/projected/a984c444-0603-4838-acc1-b99ef37c6706-kube-api-access-lzv7n\") pod \"a984c444-0603-4838-acc1-b99ef37c6706\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") "
Sep 11 16:45:45.255020 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.254791    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-home\") pod \"a984c444-0603-4838-acc1-b99ef37c6706\" (UID: \"a984c444-0603-4838-acc1-b99ef37c6706\") "
Sep 11 16:45:45.255020 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.254851    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-model-cache" (OuterVolumeSpecName: "model-cache") pod "a984c444-0603-4838-acc1-b99ef37c6706" (UID: "a984c444-0603-4838-acc1-b99ef37c6706"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:45.255020 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.254964    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a984c444-0603-4838-acc1-b99ef37c6706" (UID: "a984c444-0603-4838-acc1-b99ef37c6706"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:45.255172 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.255160    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:45.255233 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.255178    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:45.255233 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.255176    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-home" (OuterVolumeSpecName: "home") pod "a984c444-0603-4838-acc1-b99ef37c6706" (UID: "a984c444-0603-4838-acc1-b99ef37c6706"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:45.257027 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.257003    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a984c444-0603-4838-acc1-b99ef37c6706-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a984c444-0603-4838-acc1-b99ef37c6706" (UID: "a984c444-0603-4838-acc1-b99ef37c6706"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:45.257305 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.257286    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-dshm" (OuterVolumeSpecName: "dshm") pod "a984c444-0603-4838-acc1-b99ef37c6706" (UID: "a984c444-0603-4838-acc1-b99ef37c6706"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:45.257373 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.257324    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a984c444-0603-4838-acc1-b99ef37c6706-kube-api-access-lzv7n" (OuterVolumeSpecName: "kube-api-access-lzv7n") pod "a984c444-0603-4838-acc1-b99ef37c6706" (UID: "a984c444-0603-4838-acc1-b99ef37c6706"). InnerVolumeSpecName "kube-api-access-lzv7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:45:45.356447 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.356417    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:45.356447 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.356443    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a984c444-0603-4838-acc1-b99ef37c6706-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:45.356447 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.356452    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a984c444-0603-4838-acc1-b99ef37c6706-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:45.356670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.356459    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzv7n\" (UniqueName: \"kubernetes.io/projected/a984c444-0603-4838-acc1-b99ef37c6706-kube-api-access-lzv7n\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:45.439511 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.439482    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh" podUID="6987ec5e-8aea-4862-94f4-4ede669cb052" containerName="main" containerID="cri-o://cd727d1dce1bcdd654123d7fd5b9873603e9e32a3015e4cc65e7fcea38a5c3b6" gracePeriod=45
Sep 11 16:45:45.683403 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.683379    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:45:45.760001 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.759961    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-model-cache\") pod \"6987ec5e-8aea-4862-94f4-4ede669cb052\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") "
Sep 11 16:45:45.760156 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.760022    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-dshm\") pod \"6987ec5e-8aea-4862-94f4-4ede669cb052\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") "
Sep 11 16:45:45.760156 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.760074    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-home\") pod \"6987ec5e-8aea-4862-94f4-4ede669cb052\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") "
Sep 11 16:45:45.760156 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.760099    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6987ec5e-8aea-4862-94f4-4ede669cb052-tls-certs\") pod \"6987ec5e-8aea-4862-94f4-4ede669cb052\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") "
Sep 11 16:45:45.760311 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.760161    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsmkr\" (UniqueName: \"kubernetes.io/projected/6987ec5e-8aea-4862-94f4-4ede669cb052-kube-api-access-xsmkr\") pod \"6987ec5e-8aea-4862-94f4-4ede669cb052\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") "
Sep 11 16:45:45.760311 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.760186    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-model-cache" (OuterVolumeSpecName: "model-cache") pod "6987ec5e-8aea-4862-94f4-4ede669cb052" (UID: "6987ec5e-8aea-4862-94f4-4ede669cb052"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:45.760311 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.760233    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-tmp-dir\") pod \"6987ec5e-8aea-4862-94f4-4ede669cb052\" (UID: \"6987ec5e-8aea-4862-94f4-4ede669cb052\") "
Sep 11 16:45:45.760460 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.760428    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-home" (OuterVolumeSpecName: "home") pod "6987ec5e-8aea-4862-94f4-4ede669cb052" (UID: "6987ec5e-8aea-4862-94f4-4ede669cb052"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:45.760555 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.760531    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6987ec5e-8aea-4862-94f4-4ede669cb052" (UID: "6987ec5e-8aea-4862-94f4-4ede669cb052"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:45.760615 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.760558    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:45.760615 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.760572    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:45.762395 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.762369    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6987ec5e-8aea-4862-94f4-4ede669cb052-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6987ec5e-8aea-4862-94f4-4ede669cb052" (UID: "6987ec5e-8aea-4862-94f4-4ede669cb052"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:45.762395 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.762376    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-dshm" (OuterVolumeSpecName: "dshm") pod "6987ec5e-8aea-4862-94f4-4ede669cb052" (UID: "6987ec5e-8aea-4862-94f4-4ede669cb052"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:45.762526 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.762409    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6987ec5e-8aea-4862-94f4-4ede669cb052-kube-api-access-xsmkr" (OuterVolumeSpecName: "kube-api-access-xsmkr") pod "6987ec5e-8aea-4862-94f4-4ede669cb052" (UID: "6987ec5e-8aea-4862-94f4-4ede669cb052"). InnerVolumeSpecName "kube-api-access-xsmkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:45:45.796945 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.796921    2545 generic.go:356] "Generic (PLEG): container finished" podID="a984c444-0603-4838-acc1-b99ef37c6706" containerID="90c4b704dc8e9485c9d694d06b71ada9ec7d372d10f615a8036e3b6b87250a3e" exitCode=0
Sep 11 16:45:45.797051 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.796961    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"
Sep 11 16:45:45.797051 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.796992    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s" event={"ID":"a984c444-0603-4838-acc1-b99ef37c6706","Type":"ContainerDied","Data":"90c4b704dc8e9485c9d694d06b71ada9ec7d372d10f615a8036e3b6b87250a3e"}
Sep 11 16:45:45.797051 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.797031    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s" event={"ID":"a984c444-0603-4838-acc1-b99ef37c6706","Type":"ContainerDied","Data":"ced3513c6e42a0451926a010b277348ac84174d82fbeafde022f9d302a06725d"}
Sep 11 16:45:45.797051 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.797051    2545 scope.go:117] "RemoveContainer" containerID="90c4b704dc8e9485c9d694d06b71ada9ec7d372d10f615a8036e3b6b87250a3e"
Sep 11 16:45:45.798348 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.798324    2545 generic.go:356] "Generic (PLEG): container finished" podID="6987ec5e-8aea-4862-94f4-4ede669cb052" containerID="cd727d1dce1bcdd654123d7fd5b9873603e9e32a3015e4cc65e7fcea38a5c3b6" exitCode=0
Sep 11 16:45:45.798455 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.798359    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"
Sep 11 16:45:45.798455 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.798380    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh" event={"ID":"6987ec5e-8aea-4862-94f4-4ede669cb052","Type":"ContainerDied","Data":"cd727d1dce1bcdd654123d7fd5b9873603e9e32a3015e4cc65e7fcea38a5c3b6"}
Sep 11 16:45:45.798571 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.798499    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh" event={"ID":"6987ec5e-8aea-4862-94f4-4ede669cb052","Type":"ContainerDied","Data":"b56a28634394a35c36dcd2d595247af1e9f6114d08dfc9dbf6902f8a283e4600"}
Sep 11 16:45:45.806467 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.806452    2545 scope.go:117] "RemoveContainer" containerID="90c4b704dc8e9485c9d694d06b71ada9ec7d372d10f615a8036e3b6b87250a3e"
Sep 11 16:45:45.806889 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:45:45.806851    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90c4b704dc8e9485c9d694d06b71ada9ec7d372d10f615a8036e3b6b87250a3e\": container with ID starting with 90c4b704dc8e9485c9d694d06b71ada9ec7d372d10f615a8036e3b6b87250a3e not found: ID does not exist" containerID="90c4b704dc8e9485c9d694d06b71ada9ec7d372d10f615a8036e3b6b87250a3e"
Sep 11 16:45:45.806969 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.806899    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90c4b704dc8e9485c9d694d06b71ada9ec7d372d10f615a8036e3b6b87250a3e"} err="failed to get container status \"90c4b704dc8e9485c9d694d06b71ada9ec7d372d10f615a8036e3b6b87250a3e\": rpc error: code = NotFound desc = could not find container \"90c4b704dc8e9485c9d694d06b71ada9ec7d372d10f615a8036e3b6b87250a3e\": container with ID starting with 90c4b704dc8e9485c9d694d06b71ada9ec7d372d10f615a8036e3b6b87250a3e not found: ID does not exist"
Sep 11 16:45:45.806969 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.806915    2545 scope.go:117] "RemoveContainer" containerID="cd727d1dce1bcdd654123d7fd5b9873603e9e32a3015e4cc65e7fcea38a5c3b6"
Sep 11 16:45:45.815303 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.815287    2545 scope.go:117] "RemoveContainer" containerID="cd727d1dce1bcdd654123d7fd5b9873603e9e32a3015e4cc65e7fcea38a5c3b6"
Sep 11 16:45:45.815541 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:45:45.815522    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd727d1dce1bcdd654123d7fd5b9873603e9e32a3015e4cc65e7fcea38a5c3b6\": container with ID starting with cd727d1dce1bcdd654123d7fd5b9873603e9e32a3015e4cc65e7fcea38a5c3b6 not found: ID does not exist" containerID="cd727d1dce1bcdd654123d7fd5b9873603e9e32a3015e4cc65e7fcea38a5c3b6"
Sep 11 16:45:45.815597 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.815547    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd727d1dce1bcdd654123d7fd5b9873603e9e32a3015e4cc65e7fcea38a5c3b6"} err="failed to get container status \"cd727d1dce1bcdd654123d7fd5b9873603e9e32a3015e4cc65e7fcea38a5c3b6\": rpc error: code = NotFound desc = could not find container \"cd727d1dce1bcdd654123d7fd5b9873603e9e32a3015e4cc65e7fcea38a5c3b6\": container with ID starting with cd727d1dce1bcdd654123d7fd5b9873603e9e32a3015e4cc65e7fcea38a5c3b6 not found: ID does not exist"
Sep 11 16:45:45.830048 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.830028    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"]
Sep 11 16:45:45.834591 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.834572    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-0e8f65d9-kserve-74479c974d-dlb8s"]
Sep 11 16:45:45.850965 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.850946    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"]
Sep 11 16:45:45.853693 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.853675    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-0e8f65d9-kserve-84574466f4-fpwrh"]
Sep 11 16:45:45.861073 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.861057    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:45.861142 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.861077    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6987ec5e-8aea-4862-94f4-4ede669cb052-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:45.861142 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.861085    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsmkr\" (UniqueName: \"kubernetes.io/projected/6987ec5e-8aea-4862-94f4-4ede669cb052-kube-api-access-xsmkr\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:45.861142 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.861095    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6987ec5e-8aea-4862-94f4-4ede669cb052-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:45.937590 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.937570    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6987ec5e-8aea-4862-94f4-4ede669cb052" path="/var/lib/kubelet/pods/6987ec5e-8aea-4862-94f4-4ede669cb052/volumes"
Sep 11 16:45:45.937975 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:45.937961    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a984c444-0603-4838-acc1-b99ef37c6706" path="/var/lib/kubelet/pods/a984c444-0603-4838-acc1-b99ef37c6706/volumes"
Sep 11 16:45:46.948265 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:46.948230    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn" podUID="f11d7713-5c6c-4d67-a79b-0e82c98a07e0" containerName="main" containerID="cri-o://85c8d0b21d812bec1bd695fd66b8743872985e274288237723e22ec36fe93a9e" gracePeriod=45
Sep 11 16:45:47.202367 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.202316    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:47.274411 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.274376    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-tmp-dir\") pod \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") "
Sep 11 16:45:47.274571 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.274506    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-home\") pod \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") "
Sep 11 16:45:47.274571 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.274541    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-model-cache\") pod \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") "
Sep 11 16:45:47.274680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.274572    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrl45\" (UniqueName: \"kubernetes.io/projected/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-kube-api-access-xrl45\") pod \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") "
Sep 11 16:45:47.274680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.274604    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-dshm\") pod \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") "
Sep 11 16:45:47.274680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.274599    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f11d7713-5c6c-4d67-a79b-0e82c98a07e0" (UID: "f11d7713-5c6c-4d67-a79b-0e82c98a07e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:47.274829 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.274692    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-tls-certs\") pod \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\" (UID: \"f11d7713-5c6c-4d67-a79b-0e82c98a07e0\") "
Sep 11 16:45:47.274829 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.274772    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-home" (OuterVolumeSpecName: "home") pod "f11d7713-5c6c-4d67-a79b-0e82c98a07e0" (UID: "f11d7713-5c6c-4d67-a79b-0e82c98a07e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:47.274829 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.274788    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "f11d7713-5c6c-4d67-a79b-0e82c98a07e0" (UID: "f11d7713-5c6c-4d67-a79b-0e82c98a07e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:47.275084 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.275060    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:47.275084 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.275085    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:47.275422 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.275098    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:47.276968 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.276943    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f11d7713-5c6c-4d67-a79b-0e82c98a07e0" (UID: "f11d7713-5c6c-4d67-a79b-0e82c98a07e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:45:47.277055 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.276979    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-dshm" (OuterVolumeSpecName: "dshm") pod "f11d7713-5c6c-4d67-a79b-0e82c98a07e0" (UID: "f11d7713-5c6c-4d67-a79b-0e82c98a07e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:47.277095 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.277082    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-kube-api-access-xrl45" (OuterVolumeSpecName: "kube-api-access-xrl45") pod "f11d7713-5c6c-4d67-a79b-0e82c98a07e0" (UID: "f11d7713-5c6c-4d67-a79b-0e82c98a07e0"). InnerVolumeSpecName "kube-api-access-xrl45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:45:47.375731 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.375705    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrl45\" (UniqueName: \"kubernetes.io/projected/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-kube-api-access-xrl45\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:47.375731 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.375725    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:47.375731 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.375735    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f11d7713-5c6c-4d67-a79b-0e82c98a07e0-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:47.812408 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.812373    2545 generic.go:356] "Generic (PLEG): container finished" podID="f11d7713-5c6c-4d67-a79b-0e82c98a07e0" containerID="85c8d0b21d812bec1bd695fd66b8743872985e274288237723e22ec36fe93a9e" exitCode=0
Sep 11 16:45:47.812588 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.812413    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"
Sep 11 16:45:47.812588 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.812440    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn" event={"ID":"f11d7713-5c6c-4d67-a79b-0e82c98a07e0","Type":"ContainerDied","Data":"85c8d0b21d812bec1bd695fd66b8743872985e274288237723e22ec36fe93a9e"}
Sep 11 16:45:47.812588 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.812472    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn" event={"ID":"f11d7713-5c6c-4d67-a79b-0e82c98a07e0","Type":"ContainerDied","Data":"f9465647e0ddc1365a3ff1e161afaf8f29778cf3d9fdc403d0f55f3e84e3518b"}
Sep 11 16:45:47.812588 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.812487    2545 scope.go:117] "RemoveContainer" containerID="85c8d0b21d812bec1bd695fd66b8743872985e274288237723e22ec36fe93a9e"
Sep 11 16:45:47.821753 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.821734    2545 scope.go:117] "RemoveContainer" containerID="85c8d0b21d812bec1bd695fd66b8743872985e274288237723e22ec36fe93a9e"
Sep 11 16:45:47.822047 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:45:47.822030    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85c8d0b21d812bec1bd695fd66b8743872985e274288237723e22ec36fe93a9e\": container with ID starting with 85c8d0b21d812bec1bd695fd66b8743872985e274288237723e22ec36fe93a9e not found: ID does not exist" containerID="85c8d0b21d812bec1bd695fd66b8743872985e274288237723e22ec36fe93a9e"
Sep 11 16:45:47.822093 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.822056    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85c8d0b21d812bec1bd695fd66b8743872985e274288237723e22ec36fe93a9e"} err="failed to get container status \"85c8d0b21d812bec1bd695fd66b8743872985e274288237723e22ec36fe93a9e\": rpc error: code = NotFound desc = could not find container \"85c8d0b21d812bec1bd695fd66b8743872985e274288237723e22ec36fe93a9e\": container with ID starting with 85c8d0b21d812bec1bd695fd66b8743872985e274288237723e22ec36fe93a9e not found: ID does not exist"
Sep 11 16:45:47.834245 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.834222    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"]
Sep 11 16:45:47.842517 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.842500    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-42752c/autores-model-42752c-kserve-657d5c499f-qxtpn"]
Sep 11 16:45:47.937572 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:47.937540    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f11d7713-5c6c-4d67-a79b-0e82c98a07e0" path="/var/lib/kubelet/pods/f11d7713-5c6c-4d67-a79b-0e82c98a07e0/volumes"
Sep 11 16:45:51.075522 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.075490    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"]
Sep 11 16:45:51.075945 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.075722    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5" podUID="6d250f02-6408-42a9-8183-7b7bbdd723f1" containerName="istio-proxy" containerID="cri-o://01473437f7fb834e1f566a4fe4b9b6181d3d3c2401b3580d8defbcb29ebffcce" gracePeriod=30
Sep 11 16:45:51.851244 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851213    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"]
Sep 11 16:45:51.851636 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851621    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a984c444-0603-4838-acc1-b99ef37c6706" containerName="main"
Sep 11 16:45:51.851708 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851638    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="a984c444-0603-4838-acc1-b99ef37c6706" containerName="main"
Sep 11 16:45:51.851708 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851655    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6987ec5e-8aea-4862-94f4-4ede669cb052" containerName="main"
Sep 11 16:45:51.851708 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851661    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="6987ec5e-8aea-4862-94f4-4ede669cb052" containerName="main"
Sep 11 16:45:51.851708 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851668    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cf5cc26-3af8-4a30-aca4-07c3faadb98b" containerName="revoke-keys"
Sep 11 16:45:51.851708 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851674    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cf5cc26-3af8-4a30-aca4-07c3faadb98b" containerName="revoke-keys"
Sep 11 16:45:51.851708 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851706    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6530b7d2-6dc7-4792-a774-61673f74d10e" containerName="payload-pre-processing"
Sep 11 16:45:51.851708 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851711    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="6530b7d2-6dc7-4792-a774-61673f74d10e" containerName="payload-pre-processing"
Sep 11 16:45:51.852077 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851718    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f11d7713-5c6c-4d67-a79b-0e82c98a07e0" containerName="main"
Sep 11 16:45:51.852077 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851723    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="f11d7713-5c6c-4d67-a79b-0e82c98a07e0" containerName="main"
Sep 11 16:45:51.852077 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851820    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="6987ec5e-8aea-4862-94f4-4ede669cb052" containerName="main"
Sep 11 16:45:51.852077 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851829    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="a984c444-0603-4838-acc1-b99ef37c6706" containerName="main"
Sep 11 16:45:51.852077 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851836    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="f11d7713-5c6c-4d67-a79b-0e82c98a07e0" containerName="main"
Sep 11 16:45:51.852077 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851842    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cf5cc26-3af8-4a30-aca4-07c3faadb98b" containerName="revoke-keys"
Sep 11 16:45:51.852077 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.851848    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="6530b7d2-6dc7-4792-a774-61673f74d10e" containerName="payload-pre-processing"
Sep 11 16:45:51.856329 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.856306    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:51.860978 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.860957    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-a864cd-openshift-default-dockercfg-k9rt7\""
Sep 11 16:45:51.871182 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:51.871158    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"]
Sep 11 16:45:52.015107 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.015074    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-credential-socket\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.015295 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.015138    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf41047e-4495-43dd-b6e5-faa71480fffe-istiod-ca-cert\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.015295 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.015170    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx9z5\" (UniqueName: \"kubernetes.io/projected/cf41047e-4495-43dd-b6e5-faa71480fffe-kube-api-access-tx9z5\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.015411 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.015363    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-workload-socket\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.015466 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.015410    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-token\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.015517 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.015468    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-podinfo\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.015517 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.015495    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-data\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.015640 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.015562    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-workload-certs\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.015640 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.015588    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-envoy\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.116206 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.116131    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-workload-socket\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.116206 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.116169    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-token\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.116206 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.116192    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-podinfo\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.116694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.116212    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-data\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.116694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.116246    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-workload-certs\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.116694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.116265    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-envoy\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.116694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.116324    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-credential-socket\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.116694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.116363    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf41047e-4495-43dd-b6e5-faa71480fffe-istiod-ca-cert\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.116694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.116396    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx9z5\" (UniqueName: \"kubernetes.io/projected/cf41047e-4495-43dd-b6e5-faa71480fffe-kube-api-access-tx9z5\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.116694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.116565    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-workload-socket\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.116694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.116594    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-data\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.116694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.116667    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-credential-socket\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.117081 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.116792    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-workload-certs\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.117081 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.117040    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf41047e-4495-43dd-b6e5-faa71480fffe-istiod-ca-cert\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.118617 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.118595    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-envoy\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.118695 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.118653    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-podinfo\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.127843 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.127820    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-token\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.127940 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.127859    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx9z5\" (UniqueName: \"kubernetes.io/projected/cf41047e-4495-43dd-b6e5-faa71480fffe-kube-api-access-tx9z5\") pod \"e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") " pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.166503 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.166479    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:52.452211 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.452186    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"]
Sep 11 16:45:52.453472 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:45:52.453448    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf41047e_4495_43dd_b6e5_faa71480fffe.slice/crio-053f39348e47ea305177afd064d94c250204b308e938a0ba534aa6db3105cfab WatchSource:0}: Error finding container 053f39348e47ea305177afd064d94c250204b308e938a0ba534aa6db3105cfab: Status 404 returned error can't find the container with id 053f39348e47ea305177afd064d94c250204b308e938a0ba534aa6db3105cfab
Sep 11 16:45:52.455630 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.455600    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:45:52.455705 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.455667    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:45:52.455705 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.455694    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:45:52.844945 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.844902    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k" event={"ID":"cf41047e-4495-43dd-b6e5-faa71480fffe","Type":"ContainerStarted","Data":"721faa9b3251ed70be0f0dafc92712c083201ac2d1da916e3338ad73d2c0f114"}
Sep 11 16:45:52.845153 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.844976    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k" event={"ID":"cf41047e-4495-43dd-b6e5-faa71480fffe","Type":"ContainerStarted","Data":"053f39348e47ea305177afd064d94c250204b308e938a0ba534aa6db3105cfab"}
Sep 11 16:45:52.878360 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:52.878316    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k" podStartSLOduration=1.878301629 podStartE2EDuration="1.878301629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:45:52.875693504 +0000 UTC m=+971.428863479" watchObservedRunningTime="2026-09-11 16:45:52.878301629 +0000 UTC m=+971.431471554"
Sep 11 16:45:53.167046 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:53.166961    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:53.171529 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:53.171500    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:53.850391 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:53.850358    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:53.851229 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:53.851210    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:45:55.652752 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:55.652721    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:45:56.326575 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.326553    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:45:56.458338 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.458158    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-credential-socket\") pod \"6d250f02-6408-42a9-8183-7b7bbdd723f1\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") "
Sep 11 16:45:56.458338 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.458227    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-workload-socket\") pod \"6d250f02-6408-42a9-8183-7b7bbdd723f1\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") "
Sep 11 16:45:56.458653 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.458626    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d250f02-6408-42a9-8183-7b7bbdd723f1" (UID: "6d250f02-6408-42a9-8183-7b7bbdd723f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:56.458739 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.458717    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d250f02-6408-42a9-8183-7b7bbdd723f1" (UID: "6d250f02-6408-42a9-8183-7b7bbdd723f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:56.458799 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.458781    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-token\") pod \"6d250f02-6408-42a9-8183-7b7bbdd723f1\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") "
Sep 11 16:45:56.458843 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.458822    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d250f02-6408-42a9-8183-7b7bbdd723f1-istiod-ca-cert\") pod \"6d250f02-6408-42a9-8183-7b7bbdd723f1\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") "
Sep 11 16:45:56.458953 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.458938    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-workload-certs\") pod \"6d250f02-6408-42a9-8183-7b7bbdd723f1\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") "
Sep 11 16:45:56.459067 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.459052    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-envoy\") pod \"6d250f02-6408-42a9-8183-7b7bbdd723f1\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") "
Sep 11 16:45:56.459141 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.459127    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-data\") pod \"6d250f02-6408-42a9-8183-7b7bbdd723f1\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") "
Sep 11 16:45:56.459200 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.459163    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ph874\" (UniqueName: \"kubernetes.io/projected/6d250f02-6408-42a9-8183-7b7bbdd723f1-kube-api-access-ph874\") pod \"6d250f02-6408-42a9-8183-7b7bbdd723f1\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") "
Sep 11 16:45:56.459200 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.459164    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d250f02-6408-42a9-8183-7b7bbdd723f1" (UID: "6d250f02-6408-42a9-8183-7b7bbdd723f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:56.459200 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.459190    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-podinfo\") pod \"6d250f02-6408-42a9-8183-7b7bbdd723f1\" (UID: \"6d250f02-6408-42a9-8183-7b7bbdd723f1\") "
Sep 11 16:45:56.459200 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.459192    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d250f02-6408-42a9-8183-7b7bbdd723f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d250f02-6408-42a9-8183-7b7bbdd723f1" (UID: "6d250f02-6408-42a9-8183-7b7bbdd723f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:45:56.459622 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.459494    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d250f02-6408-42a9-8183-7b7bbdd723f1" (UID: "6d250f02-6408-42a9-8183-7b7bbdd723f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:56.459622 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.459546    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:56.459622 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.459567    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:56.459622 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.459583    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d250f02-6408-42a9-8183-7b7bbdd723f1-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:56.459622 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.459597    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:56.461206 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.461184    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d250f02-6408-42a9-8183-7b7bbdd723f1" (UID: "6d250f02-6408-42a9-8183-7b7bbdd723f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:45:56.461830 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.461796    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d250f02-6408-42a9-8183-7b7bbdd723f1-kube-api-access-ph874" (OuterVolumeSpecName: "kube-api-access-ph874") pod "6d250f02-6408-42a9-8183-7b7bbdd723f1" (UID: "6d250f02-6408-42a9-8183-7b7bbdd723f1"). InnerVolumeSpecName "kube-api-access-ph874". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:45:56.461937 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.461830    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d250f02-6408-42a9-8183-7b7bbdd723f1" (UID: "6d250f02-6408-42a9-8183-7b7bbdd723f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:45:56.462108 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.462088    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d250f02-6408-42a9-8183-7b7bbdd723f1" (UID: "6d250f02-6408-42a9-8183-7b7bbdd723f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:45:56.560164 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.560138    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:56.560164 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.560162    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ph874\" (UniqueName: \"kubernetes.io/projected/6d250f02-6408-42a9-8183-7b7bbdd723f1-kube-api-access-ph874\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:56.560298 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.560179    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:56.560298 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.560190    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:56.560298 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.560200    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d250f02-6408-42a9-8183-7b7bbdd723f1-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:45:56.868668 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.868639    2545 generic.go:356] "Generic (PLEG): container finished" podID="6d250f02-6408-42a9-8183-7b7bbdd723f1" containerID="01473437f7fb834e1f566a4fe4b9b6181d3d3c2401b3580d8defbcb29ebffcce" exitCode=0
Sep 11 16:45:56.869036 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.868687    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5" event={"ID":"6d250f02-6408-42a9-8183-7b7bbdd723f1","Type":"ContainerDied","Data":"01473437f7fb834e1f566a4fe4b9b6181d3d3c2401b3580d8defbcb29ebffcce"}
Sep 11 16:45:56.869036 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.868742    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5" event={"ID":"6d250f02-6408-42a9-8183-7b7bbdd723f1","Type":"ContainerDied","Data":"3d2eb2a858ffe4a299c8abd24cbb6803997c0b6b3b907cedcb2f9740b5283f71"}
Sep 11 16:45:56.869036 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.868769    2545 scope.go:117] "RemoveContainer" containerID="01473437f7fb834e1f566a4fe4b9b6181d3d3c2401b3580d8defbcb29ebffcce"
Sep 11 16:45:56.869036 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.868696    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"
Sep 11 16:45:56.878928 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.878907    2545 scope.go:117] "RemoveContainer" containerID="f4780dde6baf8b5dec3b007986d2537f12b64fc2dbc53718a13ecbb3b0261e72"
Sep 11 16:45:56.887299 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.887282    2545 scope.go:117] "RemoveContainer" containerID="01473437f7fb834e1f566a4fe4b9b6181d3d3c2401b3580d8defbcb29ebffcce"
Sep 11 16:45:56.887580 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:45:56.887564    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01473437f7fb834e1f566a4fe4b9b6181d3d3c2401b3580d8defbcb29ebffcce\": container with ID starting with 01473437f7fb834e1f566a4fe4b9b6181d3d3c2401b3580d8defbcb29ebffcce not found: ID does not exist" containerID="01473437f7fb834e1f566a4fe4b9b6181d3d3c2401b3580d8defbcb29ebffcce"
Sep 11 16:45:56.887632 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.887587    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01473437f7fb834e1f566a4fe4b9b6181d3d3c2401b3580d8defbcb29ebffcce"} err="failed to get container status \"01473437f7fb834e1f566a4fe4b9b6181d3d3c2401b3580d8defbcb29ebffcce\": rpc error: code = NotFound desc = could not find container \"01473437f7fb834e1f566a4fe4b9b6181d3d3c2401b3580d8defbcb29ebffcce\": container with ID starting with 01473437f7fb834e1f566a4fe4b9b6181d3d3c2401b3580d8defbcb29ebffcce not found: ID does not exist"
Sep 11 16:45:56.887632 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.887600    2545 scope.go:117] "RemoveContainer" containerID="f4780dde6baf8b5dec3b007986d2537f12b64fc2dbc53718a13ecbb3b0261e72"
Sep 11 16:45:56.887845 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:45:56.887827    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4780dde6baf8b5dec3b007986d2537f12b64fc2dbc53718a13ecbb3b0261e72\": container with ID starting with f4780dde6baf8b5dec3b007986d2537f12b64fc2dbc53718a13ecbb3b0261e72 not found: ID does not exist" containerID="f4780dde6baf8b5dec3b007986d2537f12b64fc2dbc53718a13ecbb3b0261e72"
Sep 11 16:45:56.887923 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.887855    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4780dde6baf8b5dec3b007986d2537f12b64fc2dbc53718a13ecbb3b0261e72"} err="failed to get container status \"f4780dde6baf8b5dec3b007986d2537f12b64fc2dbc53718a13ecbb3b0261e72\": rpc error: code = NotFound desc = could not find container \"f4780dde6baf8b5dec3b007986d2537f12b64fc2dbc53718a13ecbb3b0261e72\": container with ID starting with f4780dde6baf8b5dec3b007986d2537f12b64fc2dbc53718a13ecbb3b0261e72 not found: ID does not exist"
Sep 11 16:45:56.895237 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.895216    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"]
Sep 11 16:45:56.898557 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:56.898535    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-42752c-openshift-default-5c6dd45db8-55ml5"]
Sep 11 16:45:57.937831 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:45:57.937793    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d250f02-6408-42a9-8183-7b7bbdd723f1" path="/var/lib/kubelet/pods/6d250f02-6408-42a9-8183-7b7bbdd723f1/volumes"
Sep 11 16:46:12.671712 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:12.671677    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:46:13.209987 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.209955    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"]
Sep 11 16:46:13.210386 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.210372    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d250f02-6408-42a9-8183-7b7bbdd723f1" containerName="istio-proxy"
Sep 11 16:46:13.210431 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.210389    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d250f02-6408-42a9-8183-7b7bbdd723f1" containerName="istio-proxy"
Sep 11 16:46:13.210477 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.210468    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d250f02-6408-42a9-8183-7b7bbdd723f1" containerName="istio-proxy"
Sep 11 16:46:13.213137 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.213119    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.216001 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.215962    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-587a17\"/\"openshift-service-ca.crt\""
Sep 11 16:46:13.216001 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.215974    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-587a17\"/\"test-model-587a17-kserve-self-signed-certs\""
Sep 11 16:46:13.216237 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.216003    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-587a17\"/\"default-dockercfg-67zq5\""
Sep 11 16:46:13.216237 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.216139    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-587a17\"/\"kube-root-ca.crt\""
Sep 11 16:46:13.224394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.224374    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"]
Sep 11 16:46:13.314128 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.314099    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-dshm\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.314319 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.314137    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j44mf\" (UniqueName: \"kubernetes.io/projected/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-kube-api-access-j44mf\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.314319 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.314156    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-home\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.314319 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.314172    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-tmp-dir\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.314319 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.314229    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-model-cache\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.314319 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.314275    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-tls-certs\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.415028 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.414995    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-dshm\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.415028 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.415031    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j44mf\" (UniqueName: \"kubernetes.io/projected/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-kube-api-access-j44mf\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.415223 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.415050    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-home\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.415223 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.415067    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-tmp-dir\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.415223 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.415107    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-model-cache\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.415223 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.415169    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-tls-certs\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.415486 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.415462    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-home\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.415564 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.415527    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-tmp-dir\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.415618 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.415594    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-model-cache\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.417610 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.417586    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-dshm\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.417707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.417671    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-tls-certs\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.425313 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.425290    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j44mf\" (UniqueName: \"kubernetes.io/projected/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-kube-api-access-j44mf\") pod \"test-model-587a17-kserve-c6dfcdcff-8xm56\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") " pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.523205 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.523172    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:13.804788 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.804764    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"]
Sep 11 16:46:13.807511 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:46:13.807486    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2e759e1_03fd_47cd_bc2c_6d68116a1df6.slice/crio-cff2656a4c0a3449bad7127a324cae790bd9783b0e5e019ae361e80d5e01e283 WatchSource:0}: Error finding container cff2656a4c0a3449bad7127a324cae790bd9783b0e5e019ae361e80d5e01e283: Status 404 returned error can't find the container with id cff2656a4c0a3449bad7127a324cae790bd9783b0e5e019ae361e80d5e01e283
Sep 11 16:46:13.970526 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.970495    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56" event={"ID":"c2e759e1-03fd-47cd-bc2c-6d68116a1df6","Type":"ContainerStarted","Data":"27d675509d59e23a77bb50e05fdfcbb821d9c4862925fbddbe6345c32ffee252"}
Sep 11 16:46:13.970526 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.970533    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56" event={"ID":"c2e759e1-03fd-47cd-bc2c-6d68116a1df6","Type":"ContainerStarted","Data":"cff2656a4c0a3449bad7127a324cae790bd9783b0e5e019ae361e80d5e01e283"}
Sep 11 16:46:13.972059 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.972038    2545 generic.go:356] "Generic (PLEG): container finished" podID="cf41047e-4495-43dd-b6e5-faa71480fffe" containerID="721faa9b3251ed70be0f0dafc92712c083201ac2d1da916e3338ad73d2c0f114" exitCode=0
Sep 11 16:46:13.972155 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.972083    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k" event={"ID":"cf41047e-4495-43dd-b6e5-faa71480fffe","Type":"ContainerDied","Data":"721faa9b3251ed70be0f0dafc92712c083201ac2d1da916e3338ad73d2c0f114"}
Sep 11 16:46:13.972436 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.972423    2545 scope.go:117] "RemoveContainer" containerID="721faa9b3251ed70be0f0dafc92712c083201ac2d1da916e3338ad73d2c0f114"
Sep 11 16:46:13.973429 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.973403    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:46:13.973507 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.973466    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:46:13.973507 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.973496    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:46:13.989971 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:13.989927    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56" podStartSLOduration=0.989910631 podStartE2EDuration="989.910631ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:46:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:46:13.988262924 +0000 UTC m=+992.541432865" watchObservedRunningTime="2026-09-11 16:46:13.989910631 +0000 UTC m=+992.543080558"
Sep 11 16:46:14.983943 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:14.983065    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k" event={"ID":"cf41047e-4495-43dd-b6e5-faa71480fffe","Type":"ContainerStarted","Data":"87680153924ad3550071aff99ff99ccc5b81157afb689eb03e8f73c2ef806bea"}
Sep 11 16:46:15.167241 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:15.167210    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:46:15.167412 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:15.167346    2545 patch_prober.go:28] interesting pod/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 11 16:46:15.167412 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:15.167390    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k" podUID="cf41047e-4495-43dd-b6e5-faa71480fffe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 11 16:46:16.167652 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:16.167618    2545 patch_prober.go:28] interesting pod/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 11 16:46:16.168027 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:16.167675    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k" podUID="cf41047e-4495-43dd-b6e5-faa71480fffe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 11 16:46:17.167512 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:17.167475    2545 patch_prober.go:28] interesting pod/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 11 16:46:17.167723 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:17.167531    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k" podUID="cf41047e-4495-43dd-b6e5-faa71480fffe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 11 16:46:18.167716 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:18.167681    2545 patch_prober.go:28] interesting pod/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 11 16:46:18.168108 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:18.167732    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k" podUID="cf41047e-4495-43dd-b6e5-faa71480fffe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 11 16:46:19.167072 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:19.167042    2545 patch_prober.go:28] interesting pod/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 11 16:46:19.167239 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:19.167114    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k" podUID="cf41047e-4495-43dd-b6e5-faa71480fffe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 11 16:46:20.369016 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:20.368983    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:46:20.369422 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:20.369317    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:46:20.370100 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:20.370085    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:46:23.524152 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:23.524120    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:23.524152 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:23.524160    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:23.535977 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:23.535952    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:24.053721 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:24.053697    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:46:29.036412 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:29.036383    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:46:41.370073 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.370046    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"]
Sep 11 16:46:41.370482 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.370470    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d250f02-6408-42a9-8183-7b7bbdd723f1" containerName="istio-proxy"
Sep 11 16:46:41.370548 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.370483    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d250f02-6408-42a9-8183-7b7bbdd723f1" containerName="istio-proxy"
Sep 11 16:46:41.370586 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.370561    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d250f02-6408-42a9-8183-7b7bbdd723f1" containerName="istio-proxy"
Sep 11 16:46:41.373527 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.373511    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.389444 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.389422    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-a864cd\"/\"kube-root-ca.crt\""
Sep 11 16:46:41.389578 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.389455    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-a864cd\"/\"openshift-service-ca.crt\""
Sep 11 16:46:41.389578 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.389423    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-a864cd\"/\"explicit-model-a864cd-kserve-self-signed-certs\""
Sep 11 16:46:41.389578 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.389433    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-a864cd\"/\"default-dockercfg-4csn8\""
Sep 11 16:46:41.426576 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.426550    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"]
Sep 11 16:46:41.460474 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.460444    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-dshm\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.460589 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.460485    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abf5bcd1-0aa5-4089-beea-379cae162c2a-tls-certs\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.460589 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.460556    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-home\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.460692 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.460590    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlprj\" (UniqueName: \"kubernetes.io/projected/abf5bcd1-0aa5-4089-beea-379cae162c2a-kube-api-access-mlprj\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.460692 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.460641    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-tmp-dir\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.460781 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.460757    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-model-cache\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.562098 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.562060    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-dshm\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.562259 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.562122    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abf5bcd1-0aa5-4089-beea-379cae162c2a-tls-certs\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.562259 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.562191    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-home\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.562259 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.562224    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlprj\" (UniqueName: \"kubernetes.io/projected/abf5bcd1-0aa5-4089-beea-379cae162c2a-kube-api-access-mlprj\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.562434 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.562289    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-tmp-dir\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.562434 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.562362    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-model-cache\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.562637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.562601    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-home\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.562753 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.562738    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-tmp-dir\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.562820 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.562788    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-model-cache\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.564587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.564565    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-dshm\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.564755 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.564739    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abf5bcd1-0aa5-4089-beea-379cae162c2a-tls-certs\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.573267 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.573243    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlprj\" (UniqueName: \"kubernetes.io/projected/abf5bcd1-0aa5-4089-beea-379cae162c2a-kube-api-access-mlprj\") pod \"explicit-model-a864cd-kserve-55886c679c-jl2vj\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") " pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.683840 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.683735    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:41.963401 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:41.963320    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"]
Sep 11 16:46:41.968501 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:46:41.968475    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabf5bcd1_0aa5_4089_beea_379cae162c2a.slice/crio-48e2f37e85db9939acaf72200507fdbdb99d43e2c173dfd08674de2cbcd95004 WatchSource:0}: Error finding container 48e2f37e85db9939acaf72200507fdbdb99d43e2c173dfd08674de2cbcd95004: Status 404 returned error can't find the container with id 48e2f37e85db9939acaf72200507fdbdb99d43e2c173dfd08674de2cbcd95004
Sep 11 16:46:42.147943 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:42.147898    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj" event={"ID":"abf5bcd1-0aa5-4089-beea-379cae162c2a","Type":"ContainerStarted","Data":"ccc971a644ab9ba2908cf713367d5a2648b760f7f636278a2cabd427defb2389"}
Sep 11 16:46:42.147943 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:42.147947    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj" event={"ID":"abf5bcd1-0aa5-4089-beea-379cae162c2a","Type":"ContainerStarted","Data":"48e2f37e85db9939acaf72200507fdbdb99d43e2c173dfd08674de2cbcd95004"}
Sep 11 16:46:42.176715 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:42.176663    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj" podStartSLOduration=1.176647376 podStartE2EDuration="1.176647376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:46:42.174830292 +0000 UTC m=+1020.728000240" watchObservedRunningTime="2026-09-11 16:46:42.176647376 +0000 UTC m=+1020.729817302"
Sep 11 16:46:44.369503 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:44.369468    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:46:51.684553 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:51.684517    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:51.685148 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:51.684574    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:51.696838 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:51.696816    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:52.217853 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:52.217816    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:46:58.772669 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:58.772640    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"]
Sep 11 16:46:58.914738 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:58.914699    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk"]
Sep 11 16:46:58.919779 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:58.919757    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk"
Sep 11 16:46:58.923437 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:58.923413    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:46:58.923633 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:58.923618    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:46:58.929837 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:58.929806    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk"]
Sep 11 16:46:59.017467 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:59.017443    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f248378-c2a4-40c1-ac94-a79b4dcaf3d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk\" (UID: \"5f248378-c2a4-40c1-ac94-a79b4dcaf3d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk"
Sep 11 16:46:59.118604 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:59.118529    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f248378-c2a4-40c1-ac94-a79b4dcaf3d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk\" (UID: \"5f248378-c2a4-40c1-ac94-a79b4dcaf3d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk"
Sep 11 16:46:59.119171 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:59.119155    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f248378-c2a4-40c1-ac94-a79b4dcaf3d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk\" (UID: \"5f248378-c2a4-40c1-ac94-a79b4dcaf3d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk"
Sep 11 16:46:59.231831 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:59.231810    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk"
Sep 11 16:46:59.511328 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:46:59.511267    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk"]
Sep 11 16:46:59.514162 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:46:59.514132    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f248378_c2a4_40c1_ac94_a79b4dcaf3d8.slice/crio-84f436761def1ec436263d089262b580fc821ed406b50dd0a432f11e7a0db9af WatchSource:0}: Error finding container 84f436761def1ec436263d089262b580fc821ed406b50dd0a432f11e7a0db9af: Status 404 returned error can't find the container with id 84f436761def1ec436263d089262b580fc821ed406b50dd0a432f11e7a0db9af
Sep 11 16:47:00.250341 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:00.250306    2545 generic.go:356] "Generic (PLEG): container finished" podID="5f248378-c2a4-40c1-ac94-a79b4dcaf3d8" containerID="ab79f6940365a0b251063d266f1f1f86f13aa58af72bb12cbcd76d716e0f71f0" exitCode=0
Sep 11 16:47:00.250644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:00.250388    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk" event={"ID":"5f248378-c2a4-40c1-ac94-a79b4dcaf3d8","Type":"ContainerDied","Data":"ab79f6940365a0b251063d266f1f1f86f13aa58af72bb12cbcd76d716e0f71f0"}
Sep 11 16:47:00.250644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:00.250462    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk" event={"ID":"5f248378-c2a4-40c1-ac94-a79b4dcaf3d8","Type":"ContainerStarted","Data":"84f436761def1ec436263d089262b580fc821ed406b50dd0a432f11e7a0db9af"}
Sep 11 16:47:00.451990 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:00.451913    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:47:01.387889 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:01.387839    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk"
Sep 11 16:47:01.440372 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:01.440347    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f248378-c2a4-40c1-ac94-a79b4dcaf3d8-maas-api-service-ca\") pod \"5f248378-c2a4-40c1-ac94-a79b4dcaf3d8\" (UID: \"5f248378-c2a4-40c1-ac94-a79b4dcaf3d8\") "
Sep 11 16:47:01.440688 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:01.440666    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f248378-c2a4-40c1-ac94-a79b4dcaf3d8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f248378-c2a4-40c1-ac94-a79b4dcaf3d8" (UID: "5f248378-c2a4-40c1-ac94-a79b4dcaf3d8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:47:01.541533 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:01.541468    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f248378-c2a4-40c1-ac94-a79b4dcaf3d8-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:02.262164 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:02.262139    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk"
Sep 11 16:47:02.262320 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:02.262184    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk" event={"ID":"5f248378-c2a4-40c1-ac94-a79b4dcaf3d8","Type":"ContainerDied","Data":"84f436761def1ec436263d089262b580fc821ed406b50dd0a432f11e7a0db9af"}
Sep 11 16:47:02.262320 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:02.262228    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84f436761def1ec436263d089262b580fc821ed406b50dd0a432f11e7a0db9af"
Sep 11 16:47:05.282192 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.282159    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"]
Sep 11 16:47:05.282608 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.282594    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f248378-c2a4-40c1-ac94-a79b4dcaf3d8" containerName="revoke-keys"
Sep 11 16:47:05.282650 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.282609    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f248378-c2a4-40c1-ac94-a79b4dcaf3d8" containerName="revoke-keys"
Sep 11 16:47:05.282689 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.282674    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f248378-c2a4-40c1-ac94-a79b4dcaf3d8" containerName="revoke-keys"
Sep 11 16:47:05.285775 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.285755    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.291208 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.291182    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-424dd7d1-openshift-default-dockercfg-t8rfx\""
Sep 11 16:47:05.370734 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.370708    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-envoy\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.370851 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.370740    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istiod-ca-cert\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.370851 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.370777    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-token\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.370851 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.370833    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-data\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.371033 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.370893    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-workload-socket\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.371033 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.370925    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-podinfo\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.371033 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.370975    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x669\" (UniqueName: \"kubernetes.io/projected/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-kube-api-access-5x669\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.371033 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.371017    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-workload-certs\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.371240 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.371082    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-credential-socket\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.396553 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.396525    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"]
Sep 11 16:47:05.472100 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.472072    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-credential-socket\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.472226 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.472128    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-envoy\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.472226 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.472160    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istiod-ca-cert\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.472226 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.472195    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-token\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.472389 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.472233    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-data\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.472389 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.472358    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-workload-socket\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.472500 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.472407    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-podinfo\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.472500 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.472458    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x669\" (UniqueName: \"kubernetes.io/projected/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-kube-api-access-5x669\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.472500 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.472482    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-credential-socket\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.472661 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.472502    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-workload-certs\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.472757 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.472738    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-data\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.472893 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.472843    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-workload-certs\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.472976 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.472906    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istiod-ca-cert\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.473037 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.472973    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-workload-socket\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.474629 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.474599    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-envoy\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.474913 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.474896    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-podinfo\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.482457 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.482438    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-token\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.487035 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.487007    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x669\" (UniqueName: \"kubernetes.io/projected/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-kube-api-access-5x669\") pod \"e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") " pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.595902 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.595813    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:05.896703 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.896681    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"]
Sep 11 16:47:05.898883 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:47:05.898843    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d4e7aa3_d0d2_48d0_94f7_a8e30f9a8de3.slice/crio-f0ba927361e7ce5450698b4997d28bd6de37df2b7072a2c0c0fb585d7b642115 WatchSource:0}: Error finding container f0ba927361e7ce5450698b4997d28bd6de37df2b7072a2c0c0fb585d7b642115: Status 404 returned error can't find the container with id f0ba927361e7ce5450698b4997d28bd6de37df2b7072a2c0c0fb585d7b642115
Sep 11 16:47:05.900995 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.900966    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:47:05.901056 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.901031    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:47:05.901092 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:05.901060    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:47:06.286822 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:06.286792    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" event={"ID":"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3","Type":"ContainerStarted","Data":"521a046d6e124ff7596a836507891e1fac6c6c97f3542ab2455dcd7a9b7a1111"}
Sep 11 16:47:06.287294 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:06.286851    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" event={"ID":"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3","Type":"ContainerStarted","Data":"f0ba927361e7ce5450698b4997d28bd6de37df2b7072a2c0c0fb585d7b642115"}
Sep 11 16:47:06.312328 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:06.312281    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" podStartSLOduration=1.312266489 podStartE2EDuration="1.312266489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:47:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:47:06.308915384 +0000 UTC m=+1044.862085321" watchObservedRunningTime="2026-09-11 16:47:06.312266489 +0000 UTC m=+1044.865436414"
Sep 11 16:47:06.596355 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:06.596271    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:06.601528 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:06.601502    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:07.294104 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:07.294075    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:07.294938 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:07.294923    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:11.881051 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:11.881007    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk"]
Sep 11 16:47:11.883229 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:11.883204    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a864cd-x9fjk"]
Sep 11 16:47:11.940671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:11.940640    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f248378-c2a4-40c1-ac94-a79b4dcaf3d8" path="/var/lib/kubelet/pods/5f248378-c2a4-40c1-ac94-a79b4dcaf3d8/volumes"
Sep 11 16:47:13.791950 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:13.791891    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj" podUID="abf5bcd1-0aa5-4089-beea-379cae162c2a" containerName="main" containerID="cri-o://ccc971a644ab9ba2908cf713367d5a2648b760f7f636278a2cabd427defb2389" gracePeriod=45
Sep 11 16:47:14.043500 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.043435    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:47:14.155082 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.155057    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlprj\" (UniqueName: \"kubernetes.io/projected/abf5bcd1-0aa5-4089-beea-379cae162c2a-kube-api-access-mlprj\") pod \"abf5bcd1-0aa5-4089-beea-379cae162c2a\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") "
Sep 11 16:47:14.155297 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.155114    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-model-cache\") pod \"abf5bcd1-0aa5-4089-beea-379cae162c2a\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") "
Sep 11 16:47:14.155297 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.155185    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abf5bcd1-0aa5-4089-beea-379cae162c2a-tls-certs\") pod \"abf5bcd1-0aa5-4089-beea-379cae162c2a\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") "
Sep 11 16:47:14.155297 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.155217    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-dshm\") pod \"abf5bcd1-0aa5-4089-beea-379cae162c2a\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") "
Sep 11 16:47:14.155297 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.155240    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-tmp-dir\") pod \"abf5bcd1-0aa5-4089-beea-379cae162c2a\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") "
Sep 11 16:47:14.155525 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.155373    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-home\") pod \"abf5bcd1-0aa5-4089-beea-379cae162c2a\" (UID: \"abf5bcd1-0aa5-4089-beea-379cae162c2a\") "
Sep 11 16:47:14.155525 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.155439    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-model-cache" (OuterVolumeSpecName: "model-cache") pod "abf5bcd1-0aa5-4089-beea-379cae162c2a" (UID: "abf5bcd1-0aa5-4089-beea-379cae162c2a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:14.155525 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.155450    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "abf5bcd1-0aa5-4089-beea-379cae162c2a" (UID: "abf5bcd1-0aa5-4089-beea-379cae162c2a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:14.155666 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.155642    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-home" (OuterVolumeSpecName: "home") pod "abf5bcd1-0aa5-4089-beea-379cae162c2a" (UID: "abf5bcd1-0aa5-4089-beea-379cae162c2a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:14.155795 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.155776    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:14.155951 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.155840    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:14.156020 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.155972    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:14.157480 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.157460    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abf5bcd1-0aa5-4089-beea-379cae162c2a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "abf5bcd1-0aa5-4089-beea-379cae162c2a" (UID: "abf5bcd1-0aa5-4089-beea-379cae162c2a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:47:14.157841 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.157821    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-dshm" (OuterVolumeSpecName: "dshm") pod "abf5bcd1-0aa5-4089-beea-379cae162c2a" (UID: "abf5bcd1-0aa5-4089-beea-379cae162c2a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:14.157917 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.157835    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf5bcd1-0aa5-4089-beea-379cae162c2a-kube-api-access-mlprj" (OuterVolumeSpecName: "kube-api-access-mlprj") pod "abf5bcd1-0aa5-4089-beea-379cae162c2a" (UID: "abf5bcd1-0aa5-4089-beea-379cae162c2a"). InnerVolumeSpecName "kube-api-access-mlprj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:47:14.256809 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.256777    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abf5bcd1-0aa5-4089-beea-379cae162c2a-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:14.256809 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.256804    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abf5bcd1-0aa5-4089-beea-379cae162c2a-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:14.256984 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.256816    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlprj\" (UniqueName: \"kubernetes.io/projected/abf5bcd1-0aa5-4089-beea-379cae162c2a-kube-api-access-mlprj\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:14.340436 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.340361    2545 generic.go:356] "Generic (PLEG): container finished" podID="abf5bcd1-0aa5-4089-beea-379cae162c2a" containerID="ccc971a644ab9ba2908cf713367d5a2648b760f7f636278a2cabd427defb2389" exitCode=0
Sep 11 16:47:14.340436 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.340407    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"
Sep 11 16:47:14.340590 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.340454    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj" event={"ID":"abf5bcd1-0aa5-4089-beea-379cae162c2a","Type":"ContainerDied","Data":"ccc971a644ab9ba2908cf713367d5a2648b760f7f636278a2cabd427defb2389"}
Sep 11 16:47:14.340590 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.340490    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj" event={"ID":"abf5bcd1-0aa5-4089-beea-379cae162c2a","Type":"ContainerDied","Data":"48e2f37e85db9939acaf72200507fdbdb99d43e2c173dfd08674de2cbcd95004"}
Sep 11 16:47:14.340590 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.340506    2545 scope.go:117] "RemoveContainer" containerID="ccc971a644ab9ba2908cf713367d5a2648b760f7f636278a2cabd427defb2389"
Sep 11 16:47:14.350280 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.350264    2545 scope.go:117] "RemoveContainer" containerID="ccc971a644ab9ba2908cf713367d5a2648b760f7f636278a2cabd427defb2389"
Sep 11 16:47:14.350529 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:47:14.350512    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccc971a644ab9ba2908cf713367d5a2648b760f7f636278a2cabd427defb2389\": container with ID starting with ccc971a644ab9ba2908cf713367d5a2648b760f7f636278a2cabd427defb2389 not found: ID does not exist" containerID="ccc971a644ab9ba2908cf713367d5a2648b760f7f636278a2cabd427defb2389"
Sep 11 16:47:14.350580 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.350539    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc971a644ab9ba2908cf713367d5a2648b760f7f636278a2cabd427defb2389"} err="failed to get container status \"ccc971a644ab9ba2908cf713367d5a2648b760f7f636278a2cabd427defb2389\": rpc error: code = NotFound desc = could not find container \"ccc971a644ab9ba2908cf713367d5a2648b760f7f636278a2cabd427defb2389\": container with ID starting with ccc971a644ab9ba2908cf713367d5a2648b760f7f636278a2cabd427defb2389 not found: ID does not exist"
Sep 11 16:47:14.364790 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.364766    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"]
Sep 11 16:47:14.370399 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:14.370380    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-a864cd/explicit-model-a864cd-kserve-55886c679c-jl2vj"]
Sep 11 16:47:15.938003 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:15.937971    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abf5bcd1-0aa5-4089-beea-379cae162c2a" path="/var/lib/kubelet/pods/abf5bcd1-0aa5-4089-beea-379cae162c2a/volumes"
Sep 11 16:47:18.471369 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:18.471338    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:47:20.808572 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:20.808524    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"]
Sep 11 16:47:20.809084 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:20.808935    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k" podUID="cf41047e-4495-43dd-b6e5-faa71480fffe" containerName="istio-proxy" containerID="cri-o://87680153924ad3550071aff99ff99ccc5b81157afb689eb03e8f73c2ef806bea" gracePeriod=30
Sep 11 16:47:21.518380 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.518338    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"]
Sep 11 16:47:21.518808 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.518792    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf5bcd1-0aa5-4089-beea-379cae162c2a" containerName="main"
Sep 11 16:47:21.518808 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.518808    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf5bcd1-0aa5-4089-beea-379cae162c2a" containerName="main"
Sep 11 16:47:21.519004 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.518904    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="abf5bcd1-0aa5-4089-beea-379cae162c2a" containerName="main"
Sep 11 16:47:21.523397 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.523375    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.527069 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.526851    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-22c718-openshift-default-dockercfg-hktvc\""
Sep 11 16:47:21.541034 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.541008    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"]
Sep 11 16:47:21.625569 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.625520    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-workload-socket\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.625764 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.625629    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-data\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.625764 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.625691    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istiod-ca-cert\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.625915 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.625824    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-workload-certs\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.626025 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.626005    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-token\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.626096 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.626077    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-credential-socket\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.626155 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.626119    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sldv9\" (UniqueName: \"kubernetes.io/projected/2d3584b9-ea3c-423e-85ef-a2a181ccefed-kube-api-access-sldv9\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.626214 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.626183    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-envoy\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.626265 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.626217    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-podinfo\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.727705 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.727671    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-token\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.727936 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.727715    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-credential-socket\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.727936 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.727734    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sldv9\" (UniqueName: \"kubernetes.io/projected/2d3584b9-ea3c-423e-85ef-a2a181ccefed-kube-api-access-sldv9\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.727936 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.727768    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-envoy\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.727936 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.727796    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-podinfo\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.727936 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.727838    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-workload-socket\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.727936 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.727900    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-data\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.727936 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.727930    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istiod-ca-cert\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.728261 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.727976    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-workload-certs\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.728261 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.728175    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-credential-socket\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.728538 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.728511    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-workload-socket\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.728634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.728548    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-data\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.728634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.728569    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-workload-certs\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.728634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.728627    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istiod-ca-cert\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.730264 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.730242    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-envoy\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.730500 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.730482    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-podinfo\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.738465 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.738434    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sldv9\" (UniqueName: \"kubernetes.io/projected/2d3584b9-ea3c-423e-85ef-a2a181ccefed-kube-api-access-sldv9\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.739181 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.739157    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-token\") pod \"e2e-noten-22c718-openshift-default-757949696b-zb88t\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") " pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:21.836439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:21.836360    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:22.142654 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:22.142632    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"]
Sep 11 16:47:22.143814 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:47:22.143790    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d3584b9_ea3c_423e_85ef_a2a181ccefed.slice/crio-311a1901cc9318c4d424713be4aa69fd864483d3fb591b2b86674efa5b0468e3 WatchSource:0}: Error finding container 311a1901cc9318c4d424713be4aa69fd864483d3fb591b2b86674efa5b0468e3: Status 404 returned error can't find the container with id 311a1901cc9318c4d424713be4aa69fd864483d3fb591b2b86674efa5b0468e3
Sep 11 16:47:22.146088 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:22.146055    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:47:22.146189 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:22.146134    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:47:22.146189 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:22.146182    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:47:22.397930 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:22.397816    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t" event={"ID":"2d3584b9-ea3c-423e-85ef-a2a181ccefed","Type":"ContainerStarted","Data":"7b362f47ad5a60bf9ed81099dabb3629c4c25cae3388febe4b700a999cb6f577"}
Sep 11 16:47:22.397930 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:22.397900    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t" event={"ID":"2d3584b9-ea3c-423e-85ef-a2a181ccefed","Type":"ContainerStarted","Data":"311a1901cc9318c4d424713be4aa69fd864483d3fb591b2b86674efa5b0468e3"}
Sep 11 16:47:22.422015 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:22.421964    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t" podStartSLOduration=1.421947484 podStartE2EDuration="1.421947484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:47:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:47:22.420967508 +0000 UTC m=+1060.974137614" watchObservedRunningTime="2026-09-11 16:47:22.421947484 +0000 UTC m=+1060.975117407"
Sep 11 16:47:22.837332 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:22.837287    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:23.841751 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:23.841724    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:24.410836 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:24.410799    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:24.411735 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:24.411710    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:26.086468 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.086445    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:47:26.176955 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.176929    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf41047e-4495-43dd-b6e5-faa71480fffe-istiod-ca-cert\") pod \"cf41047e-4495-43dd-b6e5-faa71480fffe\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") "
Sep 11 16:47:26.177143 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.176965    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-envoy\") pod \"cf41047e-4495-43dd-b6e5-faa71480fffe\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") "
Sep 11 16:47:26.177143 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177007    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-podinfo\") pod \"cf41047e-4495-43dd-b6e5-faa71480fffe\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") "
Sep 11 16:47:26.177143 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177022    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-workload-certs\") pod \"cf41047e-4495-43dd-b6e5-faa71480fffe\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") "
Sep 11 16:47:26.177143 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177115    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-data\") pod \"cf41047e-4495-43dd-b6e5-faa71480fffe\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") "
Sep 11 16:47:26.177358 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177171    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tx9z5\" (UniqueName: \"kubernetes.io/projected/cf41047e-4495-43dd-b6e5-faa71480fffe-kube-api-access-tx9z5\") pod \"cf41047e-4495-43dd-b6e5-faa71480fffe\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") "
Sep 11 16:47:26.177358 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177198    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-token\") pod \"cf41047e-4495-43dd-b6e5-faa71480fffe\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") "
Sep 11 16:47:26.177358 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177223    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-workload-socket\") pod \"cf41047e-4495-43dd-b6e5-faa71480fffe\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") "
Sep 11 16:47:26.177358 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177264    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-credential-socket\") pod \"cf41047e-4495-43dd-b6e5-faa71480fffe\" (UID: \"cf41047e-4495-43dd-b6e5-faa71480fffe\") "
Sep 11 16:47:26.177358 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177323    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cf41047e-4495-43dd-b6e5-faa71480fffe" (UID: "cf41047e-4495-43dd-b6e5-faa71480fffe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:26.177610 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177405    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-data" (OuterVolumeSpecName: "istio-data") pod "cf41047e-4495-43dd-b6e5-faa71480fffe" (UID: "cf41047e-4495-43dd-b6e5-faa71480fffe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:26.177610 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177410    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf41047e-4495-43dd-b6e5-faa71480fffe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cf41047e-4495-43dd-b6e5-faa71480fffe" (UID: "cf41047e-4495-43dd-b6e5-faa71480fffe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:47:26.177727 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177645    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cf41047e-4495-43dd-b6e5-faa71480fffe" (UID: "cf41047e-4495-43dd-b6e5-faa71480fffe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:26.177815 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177701    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cf41047e-4495-43dd-b6e5-faa71480fffe" (UID: "cf41047e-4495-43dd-b6e5-faa71480fffe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:26.177910 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177822    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf41047e-4495-43dd-b6e5-faa71480fffe-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:26.177976 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177941    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:26.177976 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177961    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:26.177976 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.177974    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:26.179521 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.179494    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cf41047e-4495-43dd-b6e5-faa71480fffe" (UID: "cf41047e-4495-43dd-b6e5-faa71480fffe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:47:26.179683 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.179664    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-token" (OuterVolumeSpecName: "istio-token") pod "cf41047e-4495-43dd-b6e5-faa71480fffe" (UID: "cf41047e-4495-43dd-b6e5-faa71480fffe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:47:26.179733 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.179697    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cf41047e-4495-43dd-b6e5-faa71480fffe" (UID: "cf41047e-4495-43dd-b6e5-faa71480fffe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:26.179988 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.179972    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf41047e-4495-43dd-b6e5-faa71480fffe-kube-api-access-tx9z5" (OuterVolumeSpecName: "kube-api-access-tx9z5") pod "cf41047e-4495-43dd-b6e5-faa71480fffe" (UID: "cf41047e-4495-43dd-b6e5-faa71480fffe"). InnerVolumeSpecName "kube-api-access-tx9z5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:47:26.278660 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.278633    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:26.278660 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.278656    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx9z5\" (UniqueName: \"kubernetes.io/projected/cf41047e-4495-43dd-b6e5-faa71480fffe-kube-api-access-tx9z5\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:26.278660 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.278666    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:26.279019 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.278676    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:26.279019 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.278686    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf41047e-4495-43dd-b6e5-faa71480fffe-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:26.426713 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.426626    2545 generic.go:356] "Generic (PLEG): container finished" podID="cf41047e-4495-43dd-b6e5-faa71480fffe" containerID="87680153924ad3550071aff99ff99ccc5b81157afb689eb03e8f73c2ef806bea" exitCode=0
Sep 11 16:47:26.426713 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.426677    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"
Sep 11 16:47:26.426964 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.426709    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k" event={"ID":"cf41047e-4495-43dd-b6e5-faa71480fffe","Type":"ContainerDied","Data":"87680153924ad3550071aff99ff99ccc5b81157afb689eb03e8f73c2ef806bea"}
Sep 11 16:47:26.426964 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.426769    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k" event={"ID":"cf41047e-4495-43dd-b6e5-faa71480fffe","Type":"ContainerDied","Data":"053f39348e47ea305177afd064d94c250204b308e938a0ba534aa6db3105cfab"}
Sep 11 16:47:26.426964 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.426799    2545 scope.go:117] "RemoveContainer" containerID="87680153924ad3550071aff99ff99ccc5b81157afb689eb03e8f73c2ef806bea"
Sep 11 16:47:26.438010 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.437989    2545 scope.go:117] "RemoveContainer" containerID="721faa9b3251ed70be0f0dafc92712c083201ac2d1da916e3338ad73d2c0f114"
Sep 11 16:47:26.446785 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.446768    2545 scope.go:117] "RemoveContainer" containerID="87680153924ad3550071aff99ff99ccc5b81157afb689eb03e8f73c2ef806bea"
Sep 11 16:47:26.447107 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:47:26.447086    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87680153924ad3550071aff99ff99ccc5b81157afb689eb03e8f73c2ef806bea\": container with ID starting with 87680153924ad3550071aff99ff99ccc5b81157afb689eb03e8f73c2ef806bea not found: ID does not exist" containerID="87680153924ad3550071aff99ff99ccc5b81157afb689eb03e8f73c2ef806bea"
Sep 11 16:47:26.447171 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.447117    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87680153924ad3550071aff99ff99ccc5b81157afb689eb03e8f73c2ef806bea"} err="failed to get container status \"87680153924ad3550071aff99ff99ccc5b81157afb689eb03e8f73c2ef806bea\": rpc error: code = NotFound desc = could not find container \"87680153924ad3550071aff99ff99ccc5b81157afb689eb03e8f73c2ef806bea\": container with ID starting with 87680153924ad3550071aff99ff99ccc5b81157afb689eb03e8f73c2ef806bea not found: ID does not exist"
Sep 11 16:47:26.447171 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.447134    2545 scope.go:117] "RemoveContainer" containerID="721faa9b3251ed70be0f0dafc92712c083201ac2d1da916e3338ad73d2c0f114"
Sep 11 16:47:26.447422 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:47:26.447405    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"721faa9b3251ed70be0f0dafc92712c083201ac2d1da916e3338ad73d2c0f114\": container with ID starting with 721faa9b3251ed70be0f0dafc92712c083201ac2d1da916e3338ad73d2c0f114 not found: ID does not exist" containerID="721faa9b3251ed70be0f0dafc92712c083201ac2d1da916e3338ad73d2c0f114"
Sep 11 16:47:26.447480 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.447427    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"721faa9b3251ed70be0f0dafc92712c083201ac2d1da916e3338ad73d2c0f114"} err="failed to get container status \"721faa9b3251ed70be0f0dafc92712c083201ac2d1da916e3338ad73d2c0f114\": rpc error: code = NotFound desc = could not find container \"721faa9b3251ed70be0f0dafc92712c083201ac2d1da916e3338ad73d2c0f114\": container with ID starting with 721faa9b3251ed70be0f0dafc92712c083201ac2d1da916e3338ad73d2c0f114 not found: ID does not exist"
Sep 11 16:47:26.453224 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.453201    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"]
Sep 11 16:47:26.458840 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:26.458815    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-a864cd-openshift-default-67c9bc47b-j859k"]
Sep 11 16:47:27.937604 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:27.937576    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf41047e-4495-43dd-b6e5-faa71480fffe" path="/var/lib/kubelet/pods/cf41047e-4495-43dd-b6e5-faa71480fffe/volumes"
Sep 11 16:47:29.456484 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:29.456450    2545 generic.go:356] "Generic (PLEG): container finished" podID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerID="521a046d6e124ff7596a836507891e1fac6c6c97f3542ab2455dcd7a9b7a1111" exitCode=0
Sep 11 16:47:29.456916 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:29.456521    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" event={"ID":"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3","Type":"ContainerDied","Data":"521a046d6e124ff7596a836507891e1fac6c6c97f3542ab2455dcd7a9b7a1111"}
Sep 11 16:47:29.456979 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:29.456962    2545 scope.go:117] "RemoveContainer" containerID="521a046d6e124ff7596a836507891e1fac6c6c97f3542ab2455dcd7a9b7a1111"
Sep 11 16:47:29.458001 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:29.457975    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:47:29.458067 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:29.458037    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:47:29.458106 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:29.458076    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:47:30.465061 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.465026    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" event={"ID":"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3","Type":"ContainerStarted","Data":"000b8d6d0d8fe0105076f50cdf9928cd1f7cc11379de0f0d34b658ba207a8a96"}
Sep 11 16:47:30.596055 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.596014    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:30.596610 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.596580    2545 patch_prober.go:28] interesting pod/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 11 16:47:30.596796 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.596767    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 11 16:47:30.615558 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.615526    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"]
Sep 11 16:47:30.616041 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.616014    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf41047e-4495-43dd-b6e5-faa71480fffe" containerName="istio-proxy"
Sep 11 16:47:30.616041 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.616037    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf41047e-4495-43dd-b6e5-faa71480fffe" containerName="istio-proxy"
Sep 11 16:47:30.616242 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.616060    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf41047e-4495-43dd-b6e5-faa71480fffe" containerName="istio-proxy"
Sep 11 16:47:30.616242 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.616066    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf41047e-4495-43dd-b6e5-faa71480fffe" containerName="istio-proxy"
Sep 11 16:47:30.616242 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.616147    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf41047e-4495-43dd-b6e5-faa71480fffe" containerName="istio-proxy"
Sep 11 16:47:30.619429 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.619411    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.622522 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.622428    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-22c718\"/\"openshift-service-ca.crt\""
Sep 11 16:47:30.622522 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.622455    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-22c718\"/\"noten-model-22c718-kserve-self-signed-certs\""
Sep 11 16:47:30.622680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.622563    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-22c718\"/\"kube-root-ca.crt\""
Sep 11 16:47:30.622746 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.622729    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-22c718\"/\"default-dockercfg-fdz2s\""
Sep 11 16:47:30.633514 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.633491    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"]
Sep 11 16:47:30.723776 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.723671    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-home\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.723776 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.723739    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-model-cache\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.724019 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.723854    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-dshm\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.724019 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.723938    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7bb4966-0fc8-4598-b503-b2086d0283d9-tls-certs\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.724019 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.723961    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-tmp-dir\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.724019 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.723989    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mh7l\" (UniqueName: \"kubernetes.io/projected/f7bb4966-0fc8-4598-b503-b2086d0283d9-kube-api-access-6mh7l\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.824983 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.824946    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-home\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.825172 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.825020    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-model-cache\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.825172 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.825075    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-dshm\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.825172 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.825102    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7bb4966-0fc8-4598-b503-b2086d0283d9-tls-certs\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.825172 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.825119    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-tmp-dir\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.825172 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.825143    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mh7l\" (UniqueName: \"kubernetes.io/projected/f7bb4966-0fc8-4598-b503-b2086d0283d9-kube-api-access-6mh7l\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.825427 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.825404    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-home\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.825490 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.825436    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-model-cache\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.825548 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.825502    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-tmp-dir\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.827489 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.827465    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-dshm\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.827601 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.827584    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7bb4966-0fc8-4598-b503-b2086d0283d9-tls-certs\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.835082 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.835061    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mh7l\" (UniqueName: \"kubernetes.io/projected/f7bb4966-0fc8-4598-b503-b2086d0283d9-kube-api-access-6mh7l\") pod \"noten-model-22c718-kserve-b8cd54d4c-zjjtt\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") " pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:30.929615 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:30.929577    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:31.223471 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:31.223439    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"]
Sep 11 16:47:31.226878 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:47:31.226833    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7bb4966_0fc8_4598_b503_b2086d0283d9.slice/crio-fd665bd2f7e716fddc532f0834b54c08ed29a2ae5b0e414bfc653e68d904b88b WatchSource:0}: Error finding container fd665bd2f7e716fddc532f0834b54c08ed29a2ae5b0e414bfc653e68d904b88b: Status 404 returned error can't find the container with id fd665bd2f7e716fddc532f0834b54c08ed29a2ae5b0e414bfc653e68d904b88b
Sep 11 16:47:31.474829 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:31.474791    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt" event={"ID":"f7bb4966-0fc8-4598-b503-b2086d0283d9","Type":"ContainerStarted","Data":"3c02ca4a1b103927a4f86c0b6ed50c240fe15be3a289f02e3bfb761f684fd7a2"}
Sep 11 16:47:31.475312 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:31.474840    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt" event={"ID":"f7bb4966-0fc8-4598-b503-b2086d0283d9","Type":"ContainerStarted","Data":"fd665bd2f7e716fddc532f0834b54c08ed29a2ae5b0e414bfc653e68d904b88b"}
Sep 11 16:47:31.500631 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:31.500506    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt" podStartSLOduration=1.5004698159999998 podStartE2EDuration="1.500469816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:47:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:47:31.497318727 +0000 UTC m=+1070.050488654" watchObservedRunningTime="2026-09-11 16:47:31.500469816 +0000 UTC m=+1070.053639744"
Sep 11 16:47:31.596721 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:31.596675    2545 patch_prober.go:28] interesting pod/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 11 16:47:31.596948 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:31.596743    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 11 16:47:32.596551 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:32.596501    2545 patch_prober.go:28] interesting pod/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 11 16:47:32.597060 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:32.596577    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 11 16:47:33.597346 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:33.597228    2545 patch_prober.go:28] interesting pod/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 11 16:47:33.597346 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:33.597321    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 11 16:47:34.599285 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:34.599249    2545 patch_prober.go:28] interesting pod/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 11 16:47:34.599884 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:34.599828    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 11 16:47:34.612089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:34.612053    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"]
Sep 11 16:47:34.762142 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:34.762103    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"]
Sep 11 16:47:34.762376 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:34.762352    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t" podUID="2d3584b9-ea3c-423e-85ef-a2a181ccefed" containerName="istio-proxy" containerID="cri-o://7b362f47ad5a60bf9ed81099dabb3629c4c25cae3388febe4b700a999cb6f577" gracePeriod=30
Sep 11 16:47:35.596499 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:35.596454    2545 patch_prober.go:28] interesting pod/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 11 16:47:35.596710 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:35.596546    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 11 16:47:36.601191 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:36.601160    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:36.601588 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:36.601456    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:36.602104 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:36.602088    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:47:39.412934 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:39.412898    2545 patch_prober.go:28] interesting pod/e2e-noten-22c718-openshift-default-757949696b-zb88t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 16:47:39.413372 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:39.412965    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t" podUID="2d3584b9-ea3c-423e-85ef-a2a181ccefed" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 16:47:39.866904 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:39.866856    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"]
Sep 11 16:47:40.023662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.023601    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:40.054130 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.054083    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc"]
Sep 11 16:47:40.054556 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.054538    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d3584b9-ea3c-423e-85ef-a2a181ccefed" containerName="istio-proxy"
Sep 11 16:47:40.054633 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.054559    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d3584b9-ea3c-423e-85ef-a2a181ccefed" containerName="istio-proxy"
Sep 11 16:47:40.054737 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.054724    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf41047e-4495-43dd-b6e5-faa71480fffe" containerName="istio-proxy"
Sep 11 16:47:40.054797 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.054743    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d3584b9-ea3c-423e-85ef-a2a181ccefed" containerName="istio-proxy"
Sep 11 16:47:40.057927 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.057906    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc"
Sep 11 16:47:40.060233 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.060212    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:47:40.060334 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:47:40.060258    2545 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-135-58.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-135-58.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 11 16:47:40.082481 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.082446    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc"]
Sep 11 16:47:40.113851 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.113823    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-workload-certs\") pod \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") "
Sep 11 16:47:40.114034 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.113885    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-podinfo\") pod \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") "
Sep 11 16:47:40.114034 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.113933    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-credential-socket\") pod \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") "
Sep 11 16:47:40.114034 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.113980    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-workload-socket\") pod \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") "
Sep 11 16:47:40.114034 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114005    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-envoy\") pod \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") "
Sep 11 16:47:40.114034 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114032    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sldv9\" (UniqueName: \"kubernetes.io/projected/2d3584b9-ea3c-423e-85ef-a2a181ccefed-kube-api-access-sldv9\") pod \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") "
Sep 11 16:47:40.114301 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114093    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-data\") pod \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") "
Sep 11 16:47:40.114301 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114115    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istiod-ca-cert\") pod \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") "
Sep 11 16:47:40.114301 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114143    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-token\") pod \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\" (UID: \"2d3584b9-ea3c-423e-85ef-a2a181ccefed\") "
Sep 11 16:47:40.114301 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114137    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2d3584b9-ea3c-423e-85ef-a2a181ccefed" (UID: "2d3584b9-ea3c-423e-85ef-a2a181ccefed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:40.114301 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114219    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2d3584b9-ea3c-423e-85ef-a2a181ccefed" (UID: "2d3584b9-ea3c-423e-85ef-a2a181ccefed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:40.114301 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114297    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/390e9bdf-75a5-4c92-817a-36750138e03e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc\" (UID: \"390e9bdf-75a5-4c92-817a-36750138e03e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc"
Sep 11 16:47:40.114616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114329    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2d3584b9-ea3c-423e-85ef-a2a181ccefed" (UID: "2d3584b9-ea3c-423e-85ef-a2a181ccefed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:40.114616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114378    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-data" (OuterVolumeSpecName: "istio-data") pod "2d3584b9-ea3c-423e-85ef-a2a181ccefed" (UID: "2d3584b9-ea3c-423e-85ef-a2a181ccefed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:40.114616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114491    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2d3584b9-ea3c-423e-85ef-a2a181ccefed" (UID: "2d3584b9-ea3c-423e-85ef-a2a181ccefed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:47:40.114616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114592    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:40.114616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114611    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:40.115067 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114622    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:40.115067 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114633    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:40.115067 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.114646    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:40.116429 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.116388    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2d3584b9-ea3c-423e-85ef-a2a181ccefed" (UID: "2d3584b9-ea3c-423e-85ef-a2a181ccefed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:40.116429 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.116416    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2d3584b9-ea3c-423e-85ef-a2a181ccefed" (UID: "2d3584b9-ea3c-423e-85ef-a2a181ccefed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:47:40.116571 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.116435    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-token" (OuterVolumeSpecName: "istio-token") pod "2d3584b9-ea3c-423e-85ef-a2a181ccefed" (UID: "2d3584b9-ea3c-423e-85ef-a2a181ccefed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:47:40.116571 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.116492    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d3584b9-ea3c-423e-85ef-a2a181ccefed-kube-api-access-sldv9" (OuterVolumeSpecName: "kube-api-access-sldv9") pod "2d3584b9-ea3c-423e-85ef-a2a181ccefed" (UID: "2d3584b9-ea3c-423e-85ef-a2a181ccefed"). InnerVolumeSpecName "kube-api-access-sldv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:47:40.215795 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.215734    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/390e9bdf-75a5-4c92-817a-36750138e03e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc\" (UID: \"390e9bdf-75a5-4c92-817a-36750138e03e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc"
Sep 11 16:47:40.216020 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.215969    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:40.216020 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.215988    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:40.216020 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.216002    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sldv9\" (UniqueName: \"kubernetes.io/projected/2d3584b9-ea3c-423e-85ef-a2a181ccefed-kube-api-access-sldv9\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:40.216020 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.216016    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d3584b9-ea3c-423e-85ef-a2a181ccefed-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:40.536485 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.536453    2545 generic.go:356] "Generic (PLEG): container finished" podID="2d3584b9-ea3c-423e-85ef-a2a181ccefed" containerID="7b362f47ad5a60bf9ed81099dabb3629c4c25cae3388febe4b700a999cb6f577" exitCode=0
Sep 11 16:47:40.536989 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.536519    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"
Sep 11 16:47:40.536989 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.536537    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t" event={"ID":"2d3584b9-ea3c-423e-85ef-a2a181ccefed","Type":"ContainerDied","Data":"7b362f47ad5a60bf9ed81099dabb3629c4c25cae3388febe4b700a999cb6f577"}
Sep 11 16:47:40.536989 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.536593    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t" event={"ID":"2d3584b9-ea3c-423e-85ef-a2a181ccefed","Type":"ContainerDied","Data":"311a1901cc9318c4d424713be4aa69fd864483d3fb591b2b86674efa5b0468e3"}
Sep 11 16:47:40.536989 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.536617    2545 scope.go:117] "RemoveContainer" containerID="7b362f47ad5a60bf9ed81099dabb3629c4c25cae3388febe4b700a999cb6f577"
Sep 11 16:47:40.546489 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.546473    2545 scope.go:117] "RemoveContainer" containerID="7b362f47ad5a60bf9ed81099dabb3629c4c25cae3388febe4b700a999cb6f577"
Sep 11 16:47:40.546763 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:47:40.546740    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b362f47ad5a60bf9ed81099dabb3629c4c25cae3388febe4b700a999cb6f577\": container with ID starting with 7b362f47ad5a60bf9ed81099dabb3629c4c25cae3388febe4b700a999cb6f577 not found: ID does not exist" containerID="7b362f47ad5a60bf9ed81099dabb3629c4c25cae3388febe4b700a999cb6f577"
Sep 11 16:47:40.546831 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.546776    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b362f47ad5a60bf9ed81099dabb3629c4c25cae3388febe4b700a999cb6f577"} err="failed to get container status \"7b362f47ad5a60bf9ed81099dabb3629c4c25cae3388febe4b700a999cb6f577\": rpc error: code = NotFound desc = could not find container \"7b362f47ad5a60bf9ed81099dabb3629c4c25cae3388febe4b700a999cb6f577\": container with ID starting with 7b362f47ad5a60bf9ed81099dabb3629c4c25cae3388febe4b700a999cb6f577 not found: ID does not exist"
Sep 11 16:47:40.561955 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.561922    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"]
Sep 11 16:47:40.564703 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.564677    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-22c718-openshift-default-757949696b-zb88t"]
Sep 11 16:47:40.930106 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:40.930013    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:41.217066 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:47:41.216978    2545 configmap.go:193] Couldn't get configMap odh-ai-gateway-infra/openshift-service-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Sep 11 16:47:41.217221 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:47:41.217071    2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/390e9bdf-75a5-4c92-817a-36750138e03e-maas-api-service-ca podName:390e9bdf-75a5-4c92-817a-36750138e03e nodeName:}" failed. No retries permitted until 2026-09-11 16:47:41.717050314 +0000 UTC m=+1080.270220220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-service-ca" (UniqueName: "kubernetes.io/configmap/390e9bdf-75a5-4c92-817a-36750138e03e-maas-api-service-ca") pod "maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc" (UID: "390e9bdf-75a5-4c92-817a-36750138e03e") : failed to sync configmap cache: timed out waiting for the condition
Sep 11 16:47:41.555949 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:41.555920    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:47:41.730801 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:41.730761    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/390e9bdf-75a5-4c92-817a-36750138e03e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc\" (UID: \"390e9bdf-75a5-4c92-817a-36750138e03e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc"
Sep 11 16:47:41.731440 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:41.731417    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/390e9bdf-75a5-4c92-817a-36750138e03e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc\" (UID: \"390e9bdf-75a5-4c92-817a-36750138e03e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc"
Sep 11 16:47:41.870807 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:41.870711    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:47:41.879382 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:41.879347    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc"
Sep 11 16:47:41.943133 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:41.943097    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d3584b9-ea3c-423e-85ef-a2a181ccefed" path="/var/lib/kubelet/pods/2d3584b9-ea3c-423e-85ef-a2a181ccefed/volumes"
Sep 11 16:47:42.158420 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:42.158392    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc"]
Sep 11 16:47:42.161042 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:47:42.161012    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod390e9bdf_75a5_4c92_817a_36750138e03e.slice/crio-c734cdf5b223b0a244da3c2bb3d2befb0334ff3ba31b48c8824358ea7edb99f9 WatchSource:0}: Error finding container c734cdf5b223b0a244da3c2bb3d2befb0334ff3ba31b48c8824358ea7edb99f9: Status 404 returned error can't find the container with id c734cdf5b223b0a244da3c2bb3d2befb0334ff3ba31b48c8824358ea7edb99f9
Sep 11 16:47:42.550574 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:42.550534    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc" event={"ID":"390e9bdf-75a5-4c92-817a-36750138e03e","Type":"ContainerStarted","Data":"57068e166f319d759a2d1fcde7bfeb2595fdd5c634a21eb65db392f60fae39d8"}
Sep 11 16:47:42.550761 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:42.550632    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc" event={"ID":"390e9bdf-75a5-4c92-817a-36750138e03e","Type":"ContainerStarted","Data":"c734cdf5b223b0a244da3c2bb3d2befb0334ff3ba31b48c8824358ea7edb99f9"}
Sep 11 16:47:42.575740 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:42.575672    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc" podStartSLOduration=2.575652051 podStartE2EDuration="2.575652051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:47:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:47:42.574499892 +0000 UTC m=+1081.127669819" watchObservedRunningTime="2026-09-11 16:47:42.575652051 +0000 UTC m=+1081.128821980"
Sep 11 16:47:42.687015 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:42.686972    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc"]
Sep 11 16:47:42.690701 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:42.690676    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc"
Sep 11 16:47:42.702531 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:42.702495    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc"]
Sep 11 16:47:42.743342 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:42.743308    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/904ccb93-83e5-479a-a960-db4d509474dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc\" (UID: \"904ccb93-83e5-479a-a960-db4d509474dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc"
Sep 11 16:47:42.844986 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:42.844956    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/904ccb93-83e5-479a-a960-db4d509474dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc\" (UID: \"904ccb93-83e5-479a-a960-db4d509474dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc"
Sep 11 16:47:42.845566 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:42.845545    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/904ccb93-83e5-479a-a960-db4d509474dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc\" (UID: \"904ccb93-83e5-479a-a960-db4d509474dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc"
Sep 11 16:47:43.003528 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:43.003435    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc"
Sep 11 16:47:43.282122 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:43.282097    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc"]
Sep 11 16:47:43.284035 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:47:43.284000    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod904ccb93_83e5_479a_a960_db4d509474dd.slice/crio-00711a2ca1ad1a84c77b23a54f5d6d6b0bbdcfd68ab47e2da405dbfa62abc2e8 WatchSource:0}: Error finding container 00711a2ca1ad1a84c77b23a54f5d6d6b0bbdcfd68ab47e2da405dbfa62abc2e8: Status 404 returned error can't find the container with id 00711a2ca1ad1a84c77b23a54f5d6d6b0bbdcfd68ab47e2da405dbfa62abc2e8
Sep 11 16:47:43.557800 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:43.557711    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc" event={"ID":"904ccb93-83e5-479a-a960-db4d509474dd","Type":"ContainerStarted","Data":"a2bd52cde9a7295d3e8998d262dfe1a0bdab7592dec5796a22bf5102d1b0a2d2"}
Sep 11 16:47:43.557800 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:43.557774    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc" event={"ID":"904ccb93-83e5-479a-a960-db4d509474dd","Type":"ContainerStarted","Data":"00711a2ca1ad1a84c77b23a54f5d6d6b0bbdcfd68ab47e2da405dbfa62abc2e8"}
Sep 11 16:47:43.559275 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:43.559252    2545 generic.go:356] "Generic (PLEG): container finished" podID="390e9bdf-75a5-4c92-817a-36750138e03e" containerID="57068e166f319d759a2d1fcde7bfeb2595fdd5c634a21eb65db392f60fae39d8" exitCode=0
Sep 11 16:47:43.559373 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:43.559328    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc" event={"ID":"390e9bdf-75a5-4c92-817a-36750138e03e","Type":"ContainerDied","Data":"57068e166f319d759a2d1fcde7bfeb2595fdd5c634a21eb65db392f60fae39d8"}
Sep 11 16:47:43.591058 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:43.591016    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc" podStartSLOduration=1.591000074 podStartE2EDuration="1.591000074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:47:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:47:43.588703587 +0000 UTC m=+1082.141873524" watchObservedRunningTime="2026-09-11 16:47:43.591000074 +0000 UTC m=+1082.144170000"
Sep 11 16:47:44.567217 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:44.567182    2545 generic.go:356] "Generic (PLEG): container finished" podID="904ccb93-83e5-479a-a960-db4d509474dd" containerID="a2bd52cde9a7295d3e8998d262dfe1a0bdab7592dec5796a22bf5102d1b0a2d2" exitCode=0
Sep 11 16:47:44.567416 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:44.567269    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc" event={"ID":"904ccb93-83e5-479a-a960-db4d509474dd","Type":"ContainerDied","Data":"a2bd52cde9a7295d3e8998d262dfe1a0bdab7592dec5796a22bf5102d1b0a2d2"}
Sep 11 16:47:44.711765 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:44.711741    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc"
Sep 11 16:47:44.762960 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:44.762930    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/390e9bdf-75a5-4c92-817a-36750138e03e-maas-api-service-ca\") pod \"390e9bdf-75a5-4c92-817a-36750138e03e\" (UID: \"390e9bdf-75a5-4c92-817a-36750138e03e\") "
Sep 11 16:47:44.763242 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:44.763222    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390e9bdf-75a5-4c92-817a-36750138e03e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "390e9bdf-75a5-4c92-817a-36750138e03e" (UID: "390e9bdf-75a5-4c92-817a-36750138e03e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:47:44.864691 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:44.864613    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/390e9bdf-75a5-4c92-817a-36750138e03e-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:45.574308 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:45.574278    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc"
Sep 11 16:47:45.574492 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:45.574312    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc" event={"ID":"390e9bdf-75a5-4c92-817a-36750138e03e","Type":"ContainerDied","Data":"c734cdf5b223b0a244da3c2bb3d2befb0334ff3ba31b48c8824358ea7edb99f9"}
Sep 11 16:47:45.574492 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:45.574355    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c734cdf5b223b0a244da3c2bb3d2befb0334ff3ba31b48c8824358ea7edb99f9"
Sep 11 16:47:45.714192 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:45.714165    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc"
Sep 11 16:47:45.773485 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:45.773456    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/904ccb93-83e5-479a-a960-db4d509474dd-maas-api-service-ca\") pod \"904ccb93-83e5-479a-a960-db4d509474dd\" (UID: \"904ccb93-83e5-479a-a960-db4d509474dd\") "
Sep 11 16:47:45.773851 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:45.773831    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/904ccb93-83e5-479a-a960-db4d509474dd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "904ccb93-83e5-479a-a960-db4d509474dd" (UID: "904ccb93-83e5-479a-a960-db4d509474dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:47:45.874786 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:45.874703    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/904ccb93-83e5-479a-a960-db4d509474dd-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:46.582133 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:46.582096    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc"
Sep 11 16:47:46.582310 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:46.582130    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc" event={"ID":"904ccb93-83e5-479a-a960-db4d509474dd","Type":"ContainerDied","Data":"00711a2ca1ad1a84c77b23a54f5d6d6b0bbdcfd68ab47e2da405dbfa62abc2e8"}
Sep 11 16:47:46.582310 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:46.582180    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00711a2ca1ad1a84c77b23a54f5d6d6b0bbdcfd68ab47e2da405dbfa62abc2e8"
Sep 11 16:47:47.103557 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:47.103520    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:47:49.633955 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:49.633902    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt" podUID="f7bb4966-0fc8-4598-b503-b2086d0283d9" containerName="main" containerID="cri-o://3c02ca4a1b103927a4f86c0b6ed50c240fe15be3a289f02e3bfb761f684fd7a2" gracePeriod=45
Sep 11 16:47:49.883627 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:49.883603    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:50.016103 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.016075    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7bb4966-0fc8-4598-b503-b2086d0283d9-tls-certs\") pod \"f7bb4966-0fc8-4598-b503-b2086d0283d9\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") "
Sep 11 16:47:50.016275 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.016167    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-tmp-dir\") pod \"f7bb4966-0fc8-4598-b503-b2086d0283d9\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") "
Sep 11 16:47:50.016275 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.016201    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-model-cache\") pod \"f7bb4966-0fc8-4598-b503-b2086d0283d9\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") "
Sep 11 16:47:50.016275 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.016263    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mh7l\" (UniqueName: \"kubernetes.io/projected/f7bb4966-0fc8-4598-b503-b2086d0283d9-kube-api-access-6mh7l\") pod \"f7bb4966-0fc8-4598-b503-b2086d0283d9\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") "
Sep 11 16:47:50.016446 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.016303    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-home\") pod \"f7bb4966-0fc8-4598-b503-b2086d0283d9\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") "
Sep 11 16:47:50.016446 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.016365    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-dshm\") pod \"f7bb4966-0fc8-4598-b503-b2086d0283d9\" (UID: \"f7bb4966-0fc8-4598-b503-b2086d0283d9\") "
Sep 11 16:47:50.016546 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.016440    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f7bb4966-0fc8-4598-b503-b2086d0283d9" (UID: "f7bb4966-0fc8-4598-b503-b2086d0283d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:50.016546 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.016487    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "f7bb4966-0fc8-4598-b503-b2086d0283d9" (UID: "f7bb4966-0fc8-4598-b503-b2086d0283d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:50.016636 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.016553    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-home" (OuterVolumeSpecName: "home") pod "f7bb4966-0fc8-4598-b503-b2086d0283d9" (UID: "f7bb4966-0fc8-4598-b503-b2086d0283d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:50.016800 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.016772    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:50.016800 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.016802    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:50.017003 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.016816    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:50.018544 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.018514    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7bb4966-0fc8-4598-b503-b2086d0283d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f7bb4966-0fc8-4598-b503-b2086d0283d9" (UID: "f7bb4966-0fc8-4598-b503-b2086d0283d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:47:50.018634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.018534    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7bb4966-0fc8-4598-b503-b2086d0283d9-kube-api-access-6mh7l" (OuterVolumeSpecName: "kube-api-access-6mh7l") pod "f7bb4966-0fc8-4598-b503-b2086d0283d9" (UID: "f7bb4966-0fc8-4598-b503-b2086d0283d9"). InnerVolumeSpecName "kube-api-access-6mh7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:47:50.018634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.018577    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-dshm" (OuterVolumeSpecName: "dshm") pod "f7bb4966-0fc8-4598-b503-b2086d0283d9" (UID: "f7bb4966-0fc8-4598-b503-b2086d0283d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:50.117760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.117733    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7bb4966-0fc8-4598-b503-b2086d0283d9-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:50.117760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.117755    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7bb4966-0fc8-4598-b503-b2086d0283d9-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:50.117923 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.117766    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mh7l\" (UniqueName: \"kubernetes.io/projected/f7bb4966-0fc8-4598-b503-b2086d0283d9-kube-api-access-6mh7l\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:50.609020 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.608986    2545 generic.go:356] "Generic (PLEG): container finished" podID="f7bb4966-0fc8-4598-b503-b2086d0283d9" containerID="3c02ca4a1b103927a4f86c0b6ed50c240fe15be3a289f02e3bfb761f684fd7a2" exitCode=0
Sep 11 16:47:50.609217 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.609032    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"
Sep 11 16:47:50.609217 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.609070    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt" event={"ID":"f7bb4966-0fc8-4598-b503-b2086d0283d9","Type":"ContainerDied","Data":"3c02ca4a1b103927a4f86c0b6ed50c240fe15be3a289f02e3bfb761f684fd7a2"}
Sep 11 16:47:50.609217 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.609100    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt" event={"ID":"f7bb4966-0fc8-4598-b503-b2086d0283d9","Type":"ContainerDied","Data":"fd665bd2f7e716fddc532f0834b54c08ed29a2ae5b0e414bfc653e68d904b88b"}
Sep 11 16:47:50.609217 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.609117    2545 scope.go:117] "RemoveContainer" containerID="3c02ca4a1b103927a4f86c0b6ed50c240fe15be3a289f02e3bfb761f684fd7a2"
Sep 11 16:47:50.618675 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.618657    2545 scope.go:117] "RemoveContainer" containerID="3c02ca4a1b103927a4f86c0b6ed50c240fe15be3a289f02e3bfb761f684fd7a2"
Sep 11 16:47:50.618946 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:47:50.618927    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c02ca4a1b103927a4f86c0b6ed50c240fe15be3a289f02e3bfb761f684fd7a2\": container with ID starting with 3c02ca4a1b103927a4f86c0b6ed50c240fe15be3a289f02e3bfb761f684fd7a2 not found: ID does not exist" containerID="3c02ca4a1b103927a4f86c0b6ed50c240fe15be3a289f02e3bfb761f684fd7a2"
Sep 11 16:47:50.619014 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.618953    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c02ca4a1b103927a4f86c0b6ed50c240fe15be3a289f02e3bfb761f684fd7a2"} err="failed to get container status \"3c02ca4a1b103927a4f86c0b6ed50c240fe15be3a289f02e3bfb761f684fd7a2\": rpc error: code = NotFound desc = could not find container \"3c02ca4a1b103927a4f86c0b6ed50c240fe15be3a289f02e3bfb761f684fd7a2\": container with ID starting with 3c02ca4a1b103927a4f86c0b6ed50c240fe15be3a289f02e3bfb761f684fd7a2 not found: ID does not exist"
Sep 11 16:47:50.640849 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.640826    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"]
Sep 11 16:47:50.648141 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:50.648119    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-22c718/noten-model-22c718-kserve-b8cd54d4c-zjjtt"]
Sep 11 16:47:51.082646 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:51.082615    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4264bf-29819085-dvk5p"]
Sep 11 16:47:51.087648 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:51.087621    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-4264bf-29819085-dvk5p"]
Sep 11 16:47:51.427909 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:51.427797    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc"]
Sep 11 16:47:51.429847 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:51.429818    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4264bf-ttmmc"]
Sep 11 16:47:51.943507 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:51.943463    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="390e9bdf-75a5-4c92-817a-36750138e03e" path="/var/lib/kubelet/pods/390e9bdf-75a5-4c92-817a-36750138e03e/volumes"
Sep 11 16:47:51.944007 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:51.943888    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73a5a538-6ecc-4ed7-b381-479bac30bea4" path="/var/lib/kubelet/pods/73a5a538-6ecc-4ed7-b381-479bac30bea4/volumes"
Sep 11 16:47:51.944311 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:51.944289    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7bb4966-0fc8-4598-b503-b2086d0283d9" path="/var/lib/kubelet/pods/f7bb4966-0fc8-4598-b503-b2086d0283d9/volumes"
Sep 11 16:47:54.201769 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:54.201727    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc"]
Sep 11 16:47:54.217073 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:54.217039    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-424dd7d1-g7mfc"]
Sep 11 16:47:54.887447 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:54.887400    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56" podUID="c2e759e1-03fd-47cd-bc2c-6d68116a1df6" containerName="main" containerID="cri-o://27d675509d59e23a77bb50e05fdfcbb821d9c4862925fbddbe6345c32ffee252" gracePeriod=45
Sep 11 16:47:55.140891 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.140820    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:47:55.274279 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.274249    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-home\") pod \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") "
Sep 11 16:47:55.274625 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.274358    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-model-cache\") pod \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") "
Sep 11 16:47:55.274625 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.274407    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-dshm\") pod \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") "
Sep 11 16:47:55.274625 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.274452    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-tmp-dir\") pod \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") "
Sep 11 16:47:55.274625 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.274492    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-tls-certs\") pod \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") "
Sep 11 16:47:55.274625 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.274518    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j44mf\" (UniqueName: \"kubernetes.io/projected/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-kube-api-access-j44mf\") pod \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\" (UID: \"c2e759e1-03fd-47cd-bc2c-6d68116a1df6\") "
Sep 11 16:47:55.274625 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.274553    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-home" (OuterVolumeSpecName: "home") pod "c2e759e1-03fd-47cd-bc2c-6d68116a1df6" (UID: "c2e759e1-03fd-47cd-bc2c-6d68116a1df6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:55.274938 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.274643    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-model-cache" (OuterVolumeSpecName: "model-cache") pod "c2e759e1-03fd-47cd-bc2c-6d68116a1df6" (UID: "c2e759e1-03fd-47cd-bc2c-6d68116a1df6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:55.274938 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.274735    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c2e759e1-03fd-47cd-bc2c-6d68116a1df6" (UID: "c2e759e1-03fd-47cd-bc2c-6d68116a1df6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:55.275048 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.274944    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:55.275048 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.274958    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:55.275048 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.274969    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:55.276937 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.276909    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c2e759e1-03fd-47cd-bc2c-6d68116a1df6" (UID: "c2e759e1-03fd-47cd-bc2c-6d68116a1df6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:47:55.277086 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.276999    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-dshm" (OuterVolumeSpecName: "dshm") pod "c2e759e1-03fd-47cd-bc2c-6d68116a1df6" (UID: "c2e759e1-03fd-47cd-bc2c-6d68116a1df6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:47:55.277086 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.277052    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-kube-api-access-j44mf" (OuterVolumeSpecName: "kube-api-access-j44mf") pod "c2e759e1-03fd-47cd-bc2c-6d68116a1df6" (UID: "c2e759e1-03fd-47cd-bc2c-6d68116a1df6"). InnerVolumeSpecName "kube-api-access-j44mf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:47:55.375915 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.375896    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:55.375915 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.375915    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:55.376059 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.375926    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j44mf\" (UniqueName: \"kubernetes.io/projected/c2e759e1-03fd-47cd-bc2c-6d68116a1df6-kube-api-access-j44mf\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:47:55.641651 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.641610    2545 generic.go:356] "Generic (PLEG): container finished" podID="c2e759e1-03fd-47cd-bc2c-6d68116a1df6" containerID="27d675509d59e23a77bb50e05fdfcbb821d9c4862925fbddbe6345c32ffee252" exitCode=0
Sep 11 16:47:55.641837 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.641663    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"
Sep 11 16:47:55.641837 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.641699    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56" event={"ID":"c2e759e1-03fd-47cd-bc2c-6d68116a1df6","Type":"ContainerDied","Data":"27d675509d59e23a77bb50e05fdfcbb821d9c4862925fbddbe6345c32ffee252"}
Sep 11 16:47:55.641837 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.641746    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56" event={"ID":"c2e759e1-03fd-47cd-bc2c-6d68116a1df6","Type":"ContainerDied","Data":"cff2656a4c0a3449bad7127a324cae790bd9783b0e5e019ae361e80d5e01e283"}
Sep 11 16:47:55.641837 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.641779    2545 scope.go:117] "RemoveContainer" containerID="27d675509d59e23a77bb50e05fdfcbb821d9c4862925fbddbe6345c32ffee252"
Sep 11 16:47:55.651633 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.651614    2545 scope.go:117] "RemoveContainer" containerID="27d675509d59e23a77bb50e05fdfcbb821d9c4862925fbddbe6345c32ffee252"
Sep 11 16:47:55.651962 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:47:55.651943    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27d675509d59e23a77bb50e05fdfcbb821d9c4862925fbddbe6345c32ffee252\": container with ID starting with 27d675509d59e23a77bb50e05fdfcbb821d9c4862925fbddbe6345c32ffee252 not found: ID does not exist" containerID="27d675509d59e23a77bb50e05fdfcbb821d9c4862925fbddbe6345c32ffee252"
Sep 11 16:47:55.652008 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.651972    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27d675509d59e23a77bb50e05fdfcbb821d9c4862925fbddbe6345c32ffee252"} err="failed to get container status \"27d675509d59e23a77bb50e05fdfcbb821d9c4862925fbddbe6345c32ffee252\": rpc error: code = NotFound desc = could not find container \"27d675509d59e23a77bb50e05fdfcbb821d9c4862925fbddbe6345c32ffee252\": container with ID starting with 27d675509d59e23a77bb50e05fdfcbb821d9c4862925fbddbe6345c32ffee252 not found: ID does not exist"
Sep 11 16:47:55.673040 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.673018    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"]
Sep 11 16:47:55.679831 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.679812    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-587a17/test-model-587a17-kserve-c6dfcdcff-8xm56"]
Sep 11 16:47:55.938965 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.938892    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="904ccb93-83e5-479a-a960-db4d509474dd" path="/var/lib/kubelet/pods/904ccb93-83e5-479a-a960-db4d509474dd/volumes"
Sep 11 16:47:55.939515 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:55.939493    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2e759e1-03fd-47cd-bc2c-6d68116a1df6" path="/var/lib/kubelet/pods/c2e759e1-03fd-47cd-bc2c-6d68116a1df6/volumes"
Sep 11 16:47:56.269600 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:56.269568    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:47:59.651853 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.651819    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56"]
Sep 11 16:47:59.652231 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.652212    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="390e9bdf-75a5-4c92-817a-36750138e03e" containerName="revoke-keys"
Sep 11 16:47:59.652231 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.652224    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="390e9bdf-75a5-4c92-817a-36750138e03e" containerName="revoke-keys"
Sep 11 16:47:59.652301 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.652236    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2e759e1-03fd-47cd-bc2c-6d68116a1df6" containerName="main"
Sep 11 16:47:59.652301 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.652243    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2e759e1-03fd-47cd-bc2c-6d68116a1df6" containerName="main"
Sep 11 16:47:59.652301 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.652262    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="904ccb93-83e5-479a-a960-db4d509474dd" containerName="revoke-keys"
Sep 11 16:47:59.652301 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.652267    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="904ccb93-83e5-479a-a960-db4d509474dd" containerName="revoke-keys"
Sep 11 16:47:59.652301 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.652282    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7bb4966-0fc8-4598-b503-b2086d0283d9" containerName="main"
Sep 11 16:47:59.652301 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.652287    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7bb4966-0fc8-4598-b503-b2086d0283d9" containerName="main"
Sep 11 16:47:59.652482 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.652360    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="390e9bdf-75a5-4c92-817a-36750138e03e" containerName="revoke-keys"
Sep 11 16:47:59.652482 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.652370    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7bb4966-0fc8-4598-b503-b2086d0283d9" containerName="main"
Sep 11 16:47:59.652482 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.652376    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2e759e1-03fd-47cd-bc2c-6d68116a1df6" containerName="main"
Sep 11 16:47:59.652482 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.652381    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="904ccb93-83e5-479a-a960-db4d509474dd" containerName="revoke-keys"
Sep 11 16:47:59.656699 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.656681    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56"
Sep 11 16:47:59.659623 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.659600    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:47:59.659736 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.659683    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:47:59.668589 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.668564    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56"]
Sep 11 16:47:59.817239 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.817210    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37abd596-f390-4471-9bad-fa3724ec6a3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-587a17-cpx56\" (UID: \"37abd596-f390-4471-9bad-fa3724ec6a3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56"
Sep 11 16:47:59.918593 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.918519    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37abd596-f390-4471-9bad-fa3724ec6a3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-587a17-cpx56\" (UID: \"37abd596-f390-4471-9bad-fa3724ec6a3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56"
Sep 11 16:47:59.919131 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.919115    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37abd596-f390-4471-9bad-fa3724ec6a3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-587a17-cpx56\" (UID: \"37abd596-f390-4471-9bad-fa3724ec6a3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56"
Sep 11 16:47:59.968039 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:47:59.968010    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56"
Sep 11 16:48:00.269939 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:00.269908    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56"]
Sep 11 16:48:00.273527 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:48:00.273494    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37abd596_f390_4471_9bad_fa3724ec6a3a.slice/crio-5e932d26f59974d1342fab5595fdcaca22c0fb43a71166265e6033e17cc36856 WatchSource:0}: Error finding container 5e932d26f59974d1342fab5595fdcaca22c0fb43a71166265e6033e17cc36856: Status 404 returned error can't find the container with id 5e932d26f59974d1342fab5595fdcaca22c0fb43a71166265e6033e17cc36856
Sep 11 16:48:00.674678 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:00.674611    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56" event={"ID":"37abd596-f390-4471-9bad-fa3724ec6a3a","Type":"ContainerStarted","Data":"3da24478ab05e3ecbeebcd43ee343ecd2e9e168dc3f3ad5f217ee841c713684f"}
Sep 11 16:48:00.674678 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:00.674658    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56" event={"ID":"37abd596-f390-4471-9bad-fa3724ec6a3a","Type":"ContainerStarted","Data":"5e932d26f59974d1342fab5595fdcaca22c0fb43a71166265e6033e17cc36856"}
Sep 11 16:48:00.725192 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:00.725142    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56" podStartSLOduration=1.7251277200000001 podStartE2EDuration="1.72512772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:47:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:48:00.72392866 +0000 UTC m=+1099.277098602" watchObservedRunningTime="2026-09-11 16:48:00.72512772 +0000 UTC m=+1099.278297638"
Sep 11 16:48:01.682706 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:01.682677    2545 generic.go:356] "Generic (PLEG): container finished" podID="37abd596-f390-4471-9bad-fa3724ec6a3a" containerID="3da24478ab05e3ecbeebcd43ee343ecd2e9e168dc3f3ad5f217ee841c713684f" exitCode=0
Sep 11 16:48:01.683111 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:01.682755    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56" event={"ID":"37abd596-f390-4471-9bad-fa3724ec6a3a","Type":"ContainerDied","Data":"3da24478ab05e3ecbeebcd43ee343ecd2e9e168dc3f3ad5f217ee841c713684f"}
Sep 11 16:48:02.815584 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:02.815560    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56"
Sep 11 16:48:02.950182 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:02.950083    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37abd596-f390-4471-9bad-fa3724ec6a3a-maas-api-service-ca\") pod \"37abd596-f390-4471-9bad-fa3724ec6a3a\" (UID: \"37abd596-f390-4471-9bad-fa3724ec6a3a\") "
Sep 11 16:48:02.950432 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:02.950410    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37abd596-f390-4471-9bad-fa3724ec6a3a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "37abd596-f390-4471-9bad-fa3724ec6a3a" (UID: "37abd596-f390-4471-9bad-fa3724ec6a3a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:48:03.051759 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:03.051722    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37abd596-f390-4471-9bad-fa3724ec6a3a-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:03.688284 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:03.688249    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"]
Sep 11 16:48:03.688522 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:03.688472    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerName="istio-proxy" containerID="cri-o://000b8d6d0d8fe0105076f50cdf9928cd1f7cc11379de0f0d34b658ba207a8a96" gracePeriod=30
Sep 11 16:48:03.695103 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:03.695076    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56" event={"ID":"37abd596-f390-4471-9bad-fa3724ec6a3a","Type":"ContainerDied","Data":"5e932d26f59974d1342fab5595fdcaca22c0fb43a71166265e6033e17cc36856"}
Sep 11 16:48:03.695208 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:03.695118    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e932d26f59974d1342fab5595fdcaca22c0fb43a71166265e6033e17cc36856"
Sep 11 16:48:03.695208 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:03.695088    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56"
Sep 11 16:48:06.604420 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:06.604381    2545 patch_prober.go:28] interesting pod/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 16:48:06.604813 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:06.604447    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 16:48:08.944771 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:08.944745    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:48:09.010463 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.010435    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-data\") pod \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") "
Sep 11 16:48:09.010463 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.010469    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-envoy\") pod \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") "
Sep 11 16:48:09.010696 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.010498    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-token\") pod \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") "
Sep 11 16:48:09.010696 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.010522    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-workload-certs\") pod \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") "
Sep 11 16:48:09.010696 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.010564    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5x669\" (UniqueName: \"kubernetes.io/projected/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-kube-api-access-5x669\") pod \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") "
Sep 11 16:48:09.010696 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.010607    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-credential-socket\") pod \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") "
Sep 11 16:48:09.010696 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.010643    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-podinfo\") pod \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") "
Sep 11 16:48:09.011013 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.010705    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-workload-socket\") pod \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") "
Sep 11 16:48:09.011013 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.010744    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istiod-ca-cert\") pod \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\" (UID: \"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3\") "
Sep 11 16:48:09.011013 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.010739    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-data" (OuterVolumeSpecName: "istio-data") pod "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" (UID: "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:48:09.011013 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.010947    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" (UID: "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:48:09.011219 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.011102    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" (UID: "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:48:09.011219 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.011156    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:09.011219 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.011164    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" (UID: "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:48:09.011219 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.011177    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:09.011598 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.011221    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" (UID: "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:48:09.012913 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.012857    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-token" (OuterVolumeSpecName: "istio-token") pod "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" (UID: "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:48:09.012913 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.012884    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" (UID: "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:48:09.013345 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.013321    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" (UID: "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:48:09.013410 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.013325    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-kube-api-access-5x669" (OuterVolumeSpecName: "kube-api-access-5x669") pod "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" (UID: "0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3"). InnerVolumeSpecName "kube-api-access-5x669". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:48:09.111972 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.111938    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:09.111972 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.111964    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:09.111972 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.111974    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:09.112198 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.111983    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:09.112198 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.111990    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5x669\" (UniqueName: \"kubernetes.io/projected/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-kube-api-access-5x669\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:09.112198 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.111999    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:09.112198 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.112007    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:09.741200 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.741164    2545 generic.go:356] "Generic (PLEG): container finished" podID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerID="000b8d6d0d8fe0105076f50cdf9928cd1f7cc11379de0f0d34b658ba207a8a96" exitCode=0
Sep 11 16:48:09.741375 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.741210    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"
Sep 11 16:48:09.741375 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.741247    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" event={"ID":"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3","Type":"ContainerDied","Data":"000b8d6d0d8fe0105076f50cdf9928cd1f7cc11379de0f0d34b658ba207a8a96"}
Sep 11 16:48:09.741375 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.741302    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7" event={"ID":"0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3","Type":"ContainerDied","Data":"f0ba927361e7ce5450698b4997d28bd6de37df2b7072a2c0c0fb585d7b642115"}
Sep 11 16:48:09.741375 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.741327    2545 scope.go:117] "RemoveContainer" containerID="000b8d6d0d8fe0105076f50cdf9928cd1f7cc11379de0f0d34b658ba207a8a96"
Sep 11 16:48:09.751231 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.751215    2545 scope.go:117] "RemoveContainer" containerID="521a046d6e124ff7596a836507891e1fac6c6c97f3542ab2455dcd7a9b7a1111"
Sep 11 16:48:09.759882 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.759847    2545 scope.go:117] "RemoveContainer" containerID="000b8d6d0d8fe0105076f50cdf9928cd1f7cc11379de0f0d34b658ba207a8a96"
Sep 11 16:48:09.760206 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:48:09.760184    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"000b8d6d0d8fe0105076f50cdf9928cd1f7cc11379de0f0d34b658ba207a8a96\": container with ID starting with 000b8d6d0d8fe0105076f50cdf9928cd1f7cc11379de0f0d34b658ba207a8a96 not found: ID does not exist" containerID="000b8d6d0d8fe0105076f50cdf9928cd1f7cc11379de0f0d34b658ba207a8a96"
Sep 11 16:48:09.760279 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.760214    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"000b8d6d0d8fe0105076f50cdf9928cd1f7cc11379de0f0d34b658ba207a8a96"} err="failed to get container status \"000b8d6d0d8fe0105076f50cdf9928cd1f7cc11379de0f0d34b658ba207a8a96\": rpc error: code = NotFound desc = could not find container \"000b8d6d0d8fe0105076f50cdf9928cd1f7cc11379de0f0d34b658ba207a8a96\": container with ID starting with 000b8d6d0d8fe0105076f50cdf9928cd1f7cc11379de0f0d34b658ba207a8a96 not found: ID does not exist"
Sep 11 16:48:09.760279 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.760229    2545 scope.go:117] "RemoveContainer" containerID="521a046d6e124ff7596a836507891e1fac6c6c97f3542ab2455dcd7a9b7a1111"
Sep 11 16:48:09.760520 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:48:09.760501    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"521a046d6e124ff7596a836507891e1fac6c6c97f3542ab2455dcd7a9b7a1111\": container with ID starting with 521a046d6e124ff7596a836507891e1fac6c6c97f3542ab2455dcd7a9b7a1111 not found: ID does not exist" containerID="521a046d6e124ff7596a836507891e1fac6c6c97f3542ab2455dcd7a9b7a1111"
Sep 11 16:48:09.760568 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.760526    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"521a046d6e124ff7596a836507891e1fac6c6c97f3542ab2455dcd7a9b7a1111"} err="failed to get container status \"521a046d6e124ff7596a836507891e1fac6c6c97f3542ab2455dcd7a9b7a1111\": rpc error: code = NotFound desc = could not find container \"521a046d6e124ff7596a836507891e1fac6c6c97f3542ab2455dcd7a9b7a1111\": container with ID starting with 521a046d6e124ff7596a836507891e1fac6c6c97f3542ab2455dcd7a9b7a1111 not found: ID does not exist"
Sep 11 16:48:09.775702 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.775676    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"]
Sep 11 16:48:09.784112 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.784090    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-424dd7d1-openshift-default-7bdbd844b4-d55b7"]
Sep 11 16:48:09.938206 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:09.938173    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" path="/var/lib/kubelet/pods/0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3/volumes"
Sep 11 16:48:11.062751 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:11.062719    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56"]
Sep 11 16:48:11.078517 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:11.078485    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-587a17-cpx56"]
Sep 11 16:48:11.938129 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:11.938098    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37abd596-f390-4471-9bad-fa3724ec6a3a" path="/var/lib/kubelet/pods/37abd596-f390-4471-9bad-fa3724ec6a3a/volumes"
Sep 11 16:48:13.406374 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:13.406340    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:48:21.651835 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:21.651805    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"]
Sep 11 16:48:21.652318 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:21.652046    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b" podUID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerName="istio-proxy" containerID="cri-o://6beb42224c8bb05d59a418b882a734216700bca0bd5323858efb16344cc076b4" gracePeriod=30
Sep 11 16:48:24.976754 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:24.976717    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"]
Sep 11 16:48:24.977212 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:24.977195    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37abd596-f390-4471-9bad-fa3724ec6a3a" containerName="revoke-keys"
Sep 11 16:48:24.977255 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:24.977214    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="37abd596-f390-4471-9bad-fa3724ec6a3a" containerName="revoke-keys"
Sep 11 16:48:24.977255 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:24.977231    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerName="istio-proxy"
Sep 11 16:48:24.977255 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:24.977238    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerName="istio-proxy"
Sep 11 16:48:24.977355 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:24.977257    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerName="istio-proxy"
Sep 11 16:48:24.977355 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:24.977262    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerName="istio-proxy"
Sep 11 16:48:24.977355 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:24.977324    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerName="istio-proxy"
Sep 11 16:48:24.977355 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:24.977334    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d4e7aa3-d0d2-48d0-94f7-a8e30f9a8de3" containerName="istio-proxy"
Sep 11 16:48:24.977355 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:24.977342    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="37abd596-f390-4471-9bad-fa3724ec6a3a" containerName="revoke-keys"
Sep 11 16:48:24.980571 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:24.980554    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:24.998101 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:24.998075    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-347815-openshift-default-dockercfg-79qqd\""
Sep 11 16:48:25.016436 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.016400    2545 patch_prober.go:28] interesting pod/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 16:48:25.016600 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.016463    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b" podUID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 16:48:25.059022 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.058984    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-token\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.059214 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.059040    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istiod-ca-cert\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.059214 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.059074    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-workload-certs\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.059214 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.059098    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-credential-socket\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.059214 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.059131    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-envoy\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.059214 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.059174    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-workload-socket\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.059214 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.059213    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj6jd\" (UniqueName: \"kubernetes.io/projected/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-kube-api-access-vj6jd\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.059678 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.059275    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-podinfo\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.059678 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.059345    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-data\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.137835 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.137799    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"]
Sep 11 16:48:25.161049 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.161012    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-data\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.161220 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.161103    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-token\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.161220 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.161150    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istiod-ca-cert\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.161352 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.161330    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-workload-certs\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.161417 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.161373    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-credential-socket\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.161474 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.161423    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-envoy\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.161474 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.161463    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-workload-socket\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.161577 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.161478    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-data\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.161577 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.161493    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj6jd\" (UniqueName: \"kubernetes.io/projected/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-kube-api-access-vj6jd\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.161688 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.161574    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-podinfo\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.161688 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.161652    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-workload-certs\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.161804 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.161752    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-credential-socket\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.161952 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.161931    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istiod-ca-cert\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.162063 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.162037    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-workload-socket\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.164577 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.164546    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-envoy\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.164900 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.164849    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-podinfo\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.199431 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.199396    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-token\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.199676 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.199653    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj6jd\" (UniqueName: \"kubernetes.io/projected/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-kube-api-access-vj6jd\") pod \"e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") " pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.291739 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.291701    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:25.576254 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.576142    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"]
Sep 11 16:48:25.580326 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:48:25.580293    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8bc8b91_64bc_48bc_b3c4_0d86e1f3cfb6.slice/crio-a4796bb5d0c781a3454f3a2f372534dceb193485a0f0ccfb9ed3f1d03fb425ff WatchSource:0}: Error finding container a4796bb5d0c781a3454f3a2f372534dceb193485a0f0ccfb9ed3f1d03fb425ff: Status 404 returned error can't find the container with id a4796bb5d0c781a3454f3a2f372534dceb193485a0f0ccfb9ed3f1d03fb425ff
Sep 11 16:48:25.582575 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.582539    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:48:25.582671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.582655    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:48:25.582737 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.582722    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:48:25.835814 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.835729    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv" event={"ID":"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6","Type":"ContainerStarted","Data":"928d993fbb4507ce311b1af3a9dd7ffd48feb6ae5a08ba9b6648a6fb5b03420b"}
Sep 11 16:48:25.835814 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.835781    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv" event={"ID":"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6","Type":"ContainerStarted","Data":"a4796bb5d0c781a3454f3a2f372534dceb193485a0f0ccfb9ed3f1d03fb425ff"}
Sep 11 16:48:25.872264 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:25.872210    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv" podStartSLOduration=1.872186842 podStartE2EDuration="1.872186842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:48:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:48:25.870256145 +0000 UTC m=+1124.423426088" watchObservedRunningTime="2026-09-11 16:48:25.872186842 +0000 UTC m=+1124.425356769"
Sep 11 16:48:26.292056 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.292022    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:26.297126 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.297099    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:26.847975 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.847933    2545 generic.go:356] "Generic (PLEG): container finished" podID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerID="6beb42224c8bb05d59a418b882a734216700bca0bd5323858efb16344cc076b4" exitCode=0
Sep 11 16:48:26.848144 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.847996    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b" event={"ID":"1997c155-f9e6-44be-91ce-eafa4931d09a","Type":"ContainerDied","Data":"6beb42224c8bb05d59a418b882a734216700bca0bd5323858efb16344cc076b4"}
Sep 11 16:48:26.848144 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.848082    2545 scope.go:117] "RemoveContainer" containerID="90a9410e8458a914fca5f577268cb898cb024be63b815372dcda870f1900c29a"
Sep 11 16:48:26.848374 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.848358    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:26.849487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.849472    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:26.919953 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.919885    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:48:26.981288 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.981260    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-podinfo\") pod \"1997c155-f9e6-44be-91ce-eafa4931d09a\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") "
Sep 11 16:48:26.981288 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.981302    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1997c155-f9e6-44be-91ce-eafa4931d09a-istiod-ca-cert\") pod \"1997c155-f9e6-44be-91ce-eafa4931d09a\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") "
Sep 11 16:48:26.981528 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.981348    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-workload-socket\") pod \"1997c155-f9e6-44be-91ce-eafa4931d09a\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") "
Sep 11 16:48:26.981528 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.981388    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zb5v5\" (UniqueName: \"kubernetes.io/projected/1997c155-f9e6-44be-91ce-eafa4931d09a-kube-api-access-zb5v5\") pod \"1997c155-f9e6-44be-91ce-eafa4931d09a\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") "
Sep 11 16:48:26.981528 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.981405    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-data\") pod \"1997c155-f9e6-44be-91ce-eafa4931d09a\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") "
Sep 11 16:48:26.981528 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.981460    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-envoy\") pod \"1997c155-f9e6-44be-91ce-eafa4931d09a\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") "
Sep 11 16:48:26.981528 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.981508    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-workload-certs\") pod \"1997c155-f9e6-44be-91ce-eafa4931d09a\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") "
Sep 11 16:48:26.981789 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.981535    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-credential-socket\") pod \"1997c155-f9e6-44be-91ce-eafa4931d09a\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") "
Sep 11 16:48:26.981789 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.981568    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-token\") pod \"1997c155-f9e6-44be-91ce-eafa4931d09a\" (UID: \"1997c155-f9e6-44be-91ce-eafa4931d09a\") "
Sep 11 16:48:26.981789 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.981701    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1997c155-f9e6-44be-91ce-eafa4931d09a" (UID: "1997c155-f9e6-44be-91ce-eafa4931d09a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:48:26.981789 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.981779    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1997c155-f9e6-44be-91ce-eafa4931d09a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1997c155-f9e6-44be-91ce-eafa4931d09a" (UID: "1997c155-f9e6-44be-91ce-eafa4931d09a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:48:26.982018 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.981947    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-data" (OuterVolumeSpecName: "istio-data") pod "1997c155-f9e6-44be-91ce-eafa4931d09a" (UID: "1997c155-f9e6-44be-91ce-eafa4931d09a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:48:26.982077 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.982060    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1997c155-f9e6-44be-91ce-eafa4931d09a" (UID: "1997c155-f9e6-44be-91ce-eafa4931d09a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:48:26.982250 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.982227    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1997c155-f9e6-44be-91ce-eafa4931d09a" (UID: "1997c155-f9e6-44be-91ce-eafa4931d09a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:48:26.982765 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.982648    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1997c155-f9e6-44be-91ce-eafa4931d09a-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:26.982765 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.982673    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:26.982765 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.982687    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:26.982765 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.982701    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:26.982765 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.982720    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:26.984510 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.984477    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1997c155-f9e6-44be-91ce-eafa4931d09a" (UID: "1997c155-f9e6-44be-91ce-eafa4931d09a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:48:26.984694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.984670    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-token" (OuterVolumeSpecName: "istio-token") pod "1997c155-f9e6-44be-91ce-eafa4931d09a" (UID: "1997c155-f9e6-44be-91ce-eafa4931d09a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:48:26.984753 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.984670    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1997c155-f9e6-44be-91ce-eafa4931d09a-kube-api-access-zb5v5" (OuterVolumeSpecName: "kube-api-access-zb5v5") pod "1997c155-f9e6-44be-91ce-eafa4931d09a" (UID: "1997c155-f9e6-44be-91ce-eafa4931d09a"). InnerVolumeSpecName "kube-api-access-zb5v5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:48:26.984828 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:26.984792    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1997c155-f9e6-44be-91ce-eafa4931d09a" (UID: "1997c155-f9e6-44be-91ce-eafa4931d09a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:48:27.083233 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:27.083196    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb5v5\" (UniqueName: \"kubernetes.io/projected/1997c155-f9e6-44be-91ce-eafa4931d09a-kube-api-access-zb5v5\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:27.083233 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:27.083227    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:27.083233 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:27.083237    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:27.083233 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:27.083245    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1997c155-f9e6-44be-91ce-eafa4931d09a-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:48:27.856953 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:27.856922    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"
Sep 11 16:48:27.857408 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:27.856919    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b" event={"ID":"1997c155-f9e6-44be-91ce-eafa4931d09a","Type":"ContainerDied","Data":"9514eb8dae9b3b4c7649bd248a035af78815e94aec7ea981143bd7e45dcf3cd9"}
Sep 11 16:48:27.857408 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:27.857053    2545 scope.go:117] "RemoveContainer" containerID="6beb42224c8bb05d59a418b882a734216700bca0bd5323858efb16344cc076b4"
Sep 11 16:48:27.885483 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:27.885457    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"]
Sep 11 16:48:27.891429 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:27.891395    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-587a17-openshift-default-b654dcbc8-h787b"]
Sep 11 16:48:27.938031 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:27.938000    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1997c155-f9e6-44be-91ce-eafa4931d09a" path="/var/lib/kubelet/pods/1997c155-f9e6-44be-91ce-eafa4931d09a/volumes"
Sep 11 16:48:31.160482 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:31.160447    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:48:41.873333 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:41.873299    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:48:45.971731 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:45.971704    2545 generic.go:356] "Generic (PLEG): container finished" podID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" containerID="928d993fbb4507ce311b1af3a9dd7ffd48feb6ae5a08ba9b6648a6fb5b03420b" exitCode=0
Sep 11 16:48:45.972098 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:45.971773    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv" event={"ID":"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6","Type":"ContainerDied","Data":"928d993fbb4507ce311b1af3a9dd7ffd48feb6ae5a08ba9b6648a6fb5b03420b"}
Sep 11 16:48:45.972223 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:45.972206    2545 scope.go:117] "RemoveContainer" containerID="928d993fbb4507ce311b1af3a9dd7ffd48feb6ae5a08ba9b6648a6fb5b03420b"
Sep 11 16:48:45.973232 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:45.973207    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:48:45.973295 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:45.973273    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:48:45.973337 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:45.973304    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:48:46.981544 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:46.981499    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv" event={"ID":"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6","Type":"ContainerStarted","Data":"2e427845e3115672369b2e5176cb712e4054f6a4300f8c02a9ed79c7be33d6b4"}
Sep 11 16:48:47.292138 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:47.292103    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:47.292519 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:47.292434    2545 patch_prober.go:28] interesting pod/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 11 16:48:47.292698 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:47.292675    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv" podUID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 11 16:48:48.292257 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:48.292208    2545 patch_prober.go:28] interesting pod/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 11 16:48:48.292778 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:48.292282    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv" podUID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 11 16:48:49.292175 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:49.292131    2545 patch_prober.go:28] interesting pod/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 11 16:48:49.292374 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:49.292206    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv" podUID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 11 16:48:50.292923 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:50.292456    2545 patch_prober.go:28] interesting pod/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 11 16:48:50.292923 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:50.292520    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv" podUID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 11 16:48:51.292628 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:51.292585    2545 patch_prober.go:28] interesting pod/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 11 16:48:51.292845 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:51.292648    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv" podUID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 11 16:48:52.296388 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:52.296356    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:52.296829 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:52.296640    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:52.297705 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:52.297690    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:48:56.769692 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.769662    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"]
Sep 11 16:48:56.770121 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.770106    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerName="istio-proxy"
Sep 11 16:48:56.770167 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.770123    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerName="istio-proxy"
Sep 11 16:48:56.770215 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.770206    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerName="istio-proxy"
Sep 11 16:48:56.773364 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.773348    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.775986 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.775962    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-ce902a-openshift-default-dockercfg-w6bhv\""
Sep 11 16:48:56.788686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.788662    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"]
Sep 11 16:48:56.865412 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.865376    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-workload-socket\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.865592 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.865427    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-credential-socket\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.865592 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.865453    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-istio-data\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.865592 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.865494    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/497721e4-6796-4333-9908-be581343a615-istiod-ca-cert\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.865592 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.865560    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/497721e4-6796-4333-9908-be581343a615-istio-token\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.865762 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.865623    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-workload-certs\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.865762 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.865654    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/497721e4-6796-4333-9908-be581343a615-istio-podinfo\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.865762 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.865712    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwnl9\" (UniqueName: \"kubernetes.io/projected/497721e4-6796-4333-9908-be581343a615-kube-api-access-cwnl9\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.865762 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.865731    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-istio-envoy\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.966690 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.966641    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/497721e4-6796-4333-9908-be581343a615-istio-podinfo\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.966898 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.966708    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwnl9\" (UniqueName: \"kubernetes.io/projected/497721e4-6796-4333-9908-be581343a615-kube-api-access-cwnl9\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.966898 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.966728    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-istio-envoy\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.966898 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.966757    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-workload-socket\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.966898 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.966781    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-credential-socket\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.966898 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.966800    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-istio-data\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.966898 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.966827    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/497721e4-6796-4333-9908-be581343a615-istiod-ca-cert\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.967184 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.966901    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/497721e4-6796-4333-9908-be581343a615-istio-token\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.967184 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.966948    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-workload-certs\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.967334 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.967311    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-workload-certs\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.967409 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.967364    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-istio-data\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.967511 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.967430    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-workload-socket\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.967511 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.967459    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-credential-socket\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.967824 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.967797    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/497721e4-6796-4333-9908-be581343a615-istiod-ca-cert\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.969332 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.969310    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-istio-envoy\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.969427 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.969399    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/497721e4-6796-4333-9908-be581343a615-istio-podinfo\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.981233 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.981211    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/497721e4-6796-4333-9908-be581343a615-istio-token\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:56.981526 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:56.981510    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwnl9\" (UniqueName: \"kubernetes.io/projected/497721e4-6796-4333-9908-be581343a615-kube-api-access-cwnl9\") pod \"e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5\" (UID: \"497721e4-6796-4333-9908-be581343a615\") " pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:57.084730 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:57.084636    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:57.385395 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:57.385314    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"]
Sep 11 16:48:57.388332 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:48:57.388305    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod497721e4_6796_4333_9908_be581343a615.slice/crio-bc0bc623ea3ea5463eca3e34a897c587aa760429c1ccd5c4f2237898c87d19b9 WatchSource:0}: Error finding container bc0bc623ea3ea5463eca3e34a897c587aa760429c1ccd5c4f2237898c87d19b9: Status 404 returned error can't find the container with id bc0bc623ea3ea5463eca3e34a897c587aa760429c1ccd5c4f2237898c87d19b9
Sep 11 16:48:57.390990 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:57.390960    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:48:57.391069 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:57.391035    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:48:57.391069 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:57.391063    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:48:58.059698 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.059652    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5" event={"ID":"497721e4-6796-4333-9908-be581343a615","Type":"ContainerStarted","Data":"069f5ae77b363f162480f97d928b7e1be6b02cdec2dd4fd532381566e7512e88"}
Sep 11 16:48:58.060227 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.059750    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5" event={"ID":"497721e4-6796-4333-9908-be581343a615","Type":"ContainerStarted","Data":"bc0bc623ea3ea5463eca3e34a897c587aa760429c1ccd5c4f2237898c87d19b9"}
Sep 11 16:48:58.084888 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.084817    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:58.090141 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.090078    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5" podStartSLOduration=2.090058045 podStartE2EDuration="2.090058045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:48:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:48:58.088095861 +0000 UTC m=+1156.641265787" watchObservedRunningTime="2026-09-11 16:48:58.090058045 +0000 UTC m=+1156.643227969"
Sep 11 16:48:58.091482 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.091459    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:58.633198 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.633167    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"]
Sep 11 16:48:58.633614 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.633601    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerName="istio-proxy"
Sep 11 16:48:58.633663 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.633616    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerName="istio-proxy"
Sep 11 16:48:58.633706 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.633692    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="1997c155-f9e6-44be-91ce-eafa4931d09a" containerName="istio-proxy"
Sep 11 16:48:58.637018 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.636995    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.639017 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.638997    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-3e9c29-openshift-default-dockercfg-mbnfh\""
Sep 11 16:48:58.652582 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.652558    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"]
Sep 11 16:48:58.682171 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.682139    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-istio-envoy\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.682171 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.682178    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62c50559-360b-451a-afe0-7ac96f1c3935-istio-token\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.682401 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.682208    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm68d\" (UniqueName: \"kubernetes.io/projected/62c50559-360b-451a-afe0-7ac96f1c3935-kube-api-access-jm68d\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.682401 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.682276    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-credential-socket\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.682401 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.682352    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-istio-data\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.682587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.682414    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62c50559-360b-451a-afe0-7ac96f1c3935-istiod-ca-cert\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.682587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.682460    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-workload-socket\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.682587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.682522    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-workload-certs\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.682587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.682575    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62c50559-360b-451a-afe0-7ac96f1c3935-istio-podinfo\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.783461 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.783422    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-istio-data\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.783461 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.783459    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62c50559-360b-451a-afe0-7ac96f1c3935-istiod-ca-cert\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.783728 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.783485    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-workload-socket\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.783728 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.783518    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-workload-certs\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.783728 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.783561    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62c50559-360b-451a-afe0-7ac96f1c3935-istio-podinfo\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.783935 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.783736    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-istio-envoy\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.783935 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.783788    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62c50559-360b-451a-afe0-7ac96f1c3935-istio-token\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.783935 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.783835    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm68d\" (UniqueName: \"kubernetes.io/projected/62c50559-360b-451a-afe0-7ac96f1c3935-kube-api-access-jm68d\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.783935 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.783910    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-credential-socket\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.783935 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.783916    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-istio-data\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.784218 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.783940    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-workload-socket\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.784218 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.784027    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-workload-certs\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.784218 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.784199    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-credential-socket\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.784399 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.784371    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62c50559-360b-451a-afe0-7ac96f1c3935-istiod-ca-cert\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.786245 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.786212    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62c50559-360b-451a-afe0-7ac96f1c3935-istio-podinfo\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.786417 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.786396    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-istio-envoy\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.794095 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.794072    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62c50559-360b-451a-afe0-7ac96f1c3935-istio-token\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.794395 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.794363    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm68d\" (UniqueName: \"kubernetes.io/projected/62c50559-360b-451a-afe0-7ac96f1c3935-kube-api-access-jm68d\") pod \"e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") " pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:58.948371 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:58.948272    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:48:59.066774 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:59.066727    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:59.067660 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:59.067642    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:48:59.260595 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:59.260567    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"]
Sep 11 16:48:59.262897 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:48:59.262852    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c50559_360b_451a_afe0_7ac96f1c3935.slice/crio-892dffe6bc213d358ce14c40160b7275d8377f34fb42ef369c371c065391eb6d WatchSource:0}: Error finding container 892dffe6bc213d358ce14c40160b7275d8377f34fb42ef369c371c065391eb6d: Status 404 returned error can't find the container with id 892dffe6bc213d358ce14c40160b7275d8377f34fb42ef369c371c065391eb6d
Sep 11 16:48:59.265333 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:59.265303    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:48:59.265399 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:59.265369    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:48:59.265437 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:48:59.265401    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:49:00.074513 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:00.074476    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6" event={"ID":"62c50559-360b-451a-afe0-7ac96f1c3935","Type":"ContainerStarted","Data":"d8fb7bf6e972828264239641eab52b4b349271ded3d02efa2522e1ebe544aeec"}
Sep 11 16:49:00.074991 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:00.074535    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6" event={"ID":"62c50559-360b-451a-afe0-7ac96f1c3935","Type":"ContainerStarted","Data":"892dffe6bc213d358ce14c40160b7275d8377f34fb42ef369c371c065391eb6d"}
Sep 11 16:49:00.100317 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:00.099993    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6" podStartSLOduration=2.099969454 podStartE2EDuration="2.099969454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:48:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:00.099287333 +0000 UTC m=+1158.652457259" watchObservedRunningTime="2026-09-11 16:49:00.099969454 +0000 UTC m=+1158.653139382"
Sep 11 16:49:00.949019 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:00.948977    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:49:00.954878 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:00.954834    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:49:01.080414 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:01.080379    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:49:01.081403 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:01.081383    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:49:07.562227 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:07.562150    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:49:17.122023 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:17.121984    2545 patch_prober.go:28] interesting pod/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 16:49:17.122565 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:17.122057    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6" podUID="62c50559-360b-451a-afe0-7ac96f1c3935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 16:49:18.195724 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:18.195694    2545 generic.go:356] "Generic (PLEG): container finished" podID="497721e4-6796-4333-9908-be581343a615" containerID="069f5ae77b363f162480f97d928b7e1be6b02cdec2dd4fd532381566e7512e88" exitCode=0
Sep 11 16:49:18.196169 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:18.195774    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5" event={"ID":"497721e4-6796-4333-9908-be581343a615","Type":"ContainerDied","Data":"069f5ae77b363f162480f97d928b7e1be6b02cdec2dd4fd532381566e7512e88"}
Sep 11 16:49:18.196216 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:18.196198    2545 scope.go:117] "RemoveContainer" containerID="069f5ae77b363f162480f97d928b7e1be6b02cdec2dd4fd532381566e7512e88"
Sep 11 16:49:18.197316 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:18.197291    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:49:18.197388 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:18.197353    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:49:18.197388 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:18.197384    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:49:19.207122 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:19.207080    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5" event={"ID":"497721e4-6796-4333-9908-be581343a615","Type":"ContainerStarted","Data":"f99424747695736563e865a1a749743724067899060654944e4db97cb9303bfd"}
Sep 11 16:49:19.208766 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:19.208739    2545 generic.go:356] "Generic (PLEG): container finished" podID="62c50559-360b-451a-afe0-7ac96f1c3935" containerID="d8fb7bf6e972828264239641eab52b4b349271ded3d02efa2522e1ebe544aeec" exitCode=0
Sep 11 16:49:19.208904 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:19.208782    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6" event={"ID":"62c50559-360b-451a-afe0-7ac96f1c3935","Type":"ContainerDied","Data":"d8fb7bf6e972828264239641eab52b4b349271ded3d02efa2522e1ebe544aeec"}
Sep 11 16:49:19.209163 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:19.209148    2545 scope.go:117] "RemoveContainer" containerID="d8fb7bf6e972828264239641eab52b4b349271ded3d02efa2522e1ebe544aeec"
Sep 11 16:49:19.210249 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:19.210224    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:49:19.210317 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:19.210288    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:49:19.210353 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:19.210332    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:49:20.085214 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:20.085176    2545 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 11 16:49:20.085511 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:20.085468    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5" podUID="497721e4-6796-4333-9908-be581343a615" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 11 16:49:20.086001 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:20.085912    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:49:20.223605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:20.222796    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6" event={"ID":"62c50559-360b-451a-afe0-7ac96f1c3935","Type":"ContainerStarted","Data":"228cc4ee3e0b24f3b65b5d32759746298b4bb5c83d958aa30a5eab30aa0f3f77"}
Sep 11 16:49:20.948835 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:20.948794    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:49:20.949037 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:20.948984    2545 patch_prober.go:28] interesting pod/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 11 16:49:20.949082 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:20.949039    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6" podUID="62c50559-360b-451a-afe0-7ac96f1c3935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 11 16:49:21.085526 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:21.085487    2545 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 11 16:49:21.085840 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:21.085806    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5" podUID="497721e4-6796-4333-9908-be581343a615" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 11 16:49:21.949439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:21.949407    2545 patch_prober.go:28] interesting pod/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 11 16:49:21.949925 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:21.949467    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6" podUID="62c50559-360b-451a-afe0-7ac96f1c3935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 11 16:49:22.086163 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:22.086098    2545 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 11 16:49:22.086369 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:22.086190    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5" podUID="497721e4-6796-4333-9908-be581343a615" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 11 16:49:22.948812 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:22.948780    2545 patch_prober.go:28] interesting pod/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 11 16:49:22.949096 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:22.949071    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6" podUID="62c50559-360b-451a-afe0-7ac96f1c3935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 11 16:49:23.085158 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:23.085124    2545 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 11 16:49:23.085747 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:23.085719    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5" podUID="497721e4-6796-4333-9908-be581343a615" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 11 16:49:23.949182 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:23.949147    2545 patch_prober.go:28] interesting pod/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 11 16:49:23.949356 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:23.949200    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6" podUID="62c50559-360b-451a-afe0-7ac96f1c3935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 11 16:49:24.086098 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:24.086063    2545 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 11 16:49:24.086671 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:24.086641    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5" podUID="497721e4-6796-4333-9908-be581343a615" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 11 16:49:24.949347 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:24.949311    2545 patch_prober.go:28] interesting pod/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 11 16:49:24.949529 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:24.949372    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6" podUID="62c50559-360b-451a-afe0-7ac96f1c3935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 11 16:49:25.093933 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:25.093897    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:49:25.094757 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:25.094732    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:49:25.099229 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:25.099214    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 16:49:25.953122 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:25.953090    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:49:25.953386 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:25.953367    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:49:25.954079 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:25.954058    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:49:27.855231 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:27.855203    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7780-fc6d45"]
Sep 11 16:49:27.858817 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:27.858801    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-fc6d45"
Sep 11 16:49:27.862334 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:27.862313    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vtvd2\""
Sep 11 16:49:27.871631 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:27.871609    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7780-fc6d45"]
Sep 11 16:49:27.894288 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:27.894262    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lgh4\" (UniqueName: \"kubernetes.io/projected/b5bd8d36-a276-4d3b-8109-b1a10c13465f-kube-api-access-5lgh4\") pod \"test-curl-7780-fc6d45\" (UID: \"b5bd8d36-a276-4d3b-8109-b1a10c13465f\") " pod="opendatahub/test-curl-7780-fc6d45"
Sep 11 16:49:27.995178 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:27.995155    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lgh4\" (UniqueName: \"kubernetes.io/projected/b5bd8d36-a276-4d3b-8109-b1a10c13465f-kube-api-access-5lgh4\") pod \"test-curl-7780-fc6d45\" (UID: \"b5bd8d36-a276-4d3b-8109-b1a10c13465f\") " pod="opendatahub/test-curl-7780-fc6d45"
Sep 11 16:49:28.004827 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:28.004800    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lgh4\" (UniqueName: \"kubernetes.io/projected/b5bd8d36-a276-4d3b-8109-b1a10c13465f-kube-api-access-5lgh4\") pod \"test-curl-7780-fc6d45\" (UID: \"b5bd8d36-a276-4d3b-8109-b1a10c13465f\") " pod="opendatahub/test-curl-7780-fc6d45"
Sep 11 16:49:28.168271 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:28.168215    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-fc6d45"
Sep 11 16:49:28.449511 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:28.449325    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7780-fc6d45"]
Sep 11 16:49:28.452224 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:49:28.452184    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5bd8d36_a276_4d3b_8109_b1a10c13465f.slice/crio-4ffb9cc3c82851fd5fac3f9d25c1fe28ad7dce41984df05e5b3d244be8469601 WatchSource:0}: Error finding container 4ffb9cc3c82851fd5fac3f9d25c1fe28ad7dce41984df05e5b3d244be8469601: Status 404 returned error can't find the container with id 4ffb9cc3c82851fd5fac3f9d25c1fe28ad7dce41984df05e5b3d244be8469601
Sep 11 16:49:29.278577 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:29.278540    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7780-fc6d45" event={"ID":"b5bd8d36-a276-4d3b-8109-b1a10c13465f","Type":"ContainerStarted","Data":"b50bdf5306434a718d481705f0d983a95038281b1951574cd8b102043eaa4290"}
Sep 11 16:49:29.278995 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:29.278590    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7780-fc6d45" event={"ID":"b5bd8d36-a276-4d3b-8109-b1a10c13465f","Type":"ContainerStarted","Data":"4ffb9cc3c82851fd5fac3f9d25c1fe28ad7dce41984df05e5b3d244be8469601"}
Sep 11 16:49:29.329350 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:29.329303    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7780-fc6d45" podStartSLOduration=2.329289968 podStartE2EDuration="2.329289968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:49:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:29.328386711 +0000 UTC m=+1187.881556638" watchObservedRunningTime="2026-09-11 16:49:29.329289968 +0000 UTC m=+1187.882459895"
Sep 11 16:49:30.036405 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:30.036321    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7780-fc6d45"]
Sep 11 16:49:30.043859 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:30.043831    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7780-fc6d45"]
Sep 11 16:49:30.169794 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:30.169760    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7780-72ebe9"]
Sep 11 16:49:30.170233 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:30.170218    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5bd8d36-a276-4d3b-8109-b1a10c13465f" containerName="test-curl-7780-fc6d45"
Sep 11 16:49:30.170282 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:30.170236    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5bd8d36-a276-4d3b-8109-b1a10c13465f" containerName="test-curl-7780-fc6d45"
Sep 11 16:49:30.170328 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:30.170319    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5bd8d36-a276-4d3b-8109-b1a10c13465f" containerName="test-curl-7780-fc6d45"
Sep 11 16:49:30.173708 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:30.173692    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-72ebe9"
Sep 11 16:49:30.183382 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:30.183356    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7780-72ebe9"]
Sep 11 16:49:30.215716 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:30.215695    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhx4g\" (UniqueName: \"kubernetes.io/projected/9d709ec6-2593-4929-a238-f95779370d3f-kube-api-access-hhx4g\") pod \"test-curl-7780-72ebe9\" (UID: \"9d709ec6-2593-4929-a238-f95779370d3f\") " pod="opendatahub/test-curl-7780-72ebe9"
Sep 11 16:49:30.284937 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:30.284909    2545 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7780-fc6d45" container="test-curl-7780-fc6d45"
Sep 11 16:49:30.316577 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:30.316513    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhx4g\" (UniqueName: \"kubernetes.io/projected/9d709ec6-2593-4929-a238-f95779370d3f-kube-api-access-hhx4g\") pod \"test-curl-7780-72ebe9\" (UID: \"9d709ec6-2593-4929-a238-f95779370d3f\") " pod="opendatahub/test-curl-7780-72ebe9"
Sep 11 16:49:30.328210 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:30.328187    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhx4g\" (UniqueName: \"kubernetes.io/projected/9d709ec6-2593-4929-a238-f95779370d3f-kube-api-access-hhx4g\") pod \"test-curl-7780-72ebe9\" (UID: \"9d709ec6-2593-4929-a238-f95779370d3f\") " pod="opendatahub/test-curl-7780-72ebe9"
Sep 11 16:49:30.483938 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:30.483915    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-72ebe9"
Sep 11 16:49:30.773704 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:30.773677    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7780-72ebe9"]
Sep 11 16:49:30.778236 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:49:30.778212    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d709ec6_2593_4929_a238_f95779370d3f.slice/crio-cb70b980184ba4d857abc9fd4beade1d1e2e8b5f676bf84c518f2e01d2a4a836 WatchSource:0}: Error finding container cb70b980184ba4d857abc9fd4beade1d1e2e8b5f676bf84c518f2e01d2a4a836: Status 404 returned error can't find the container with id cb70b980184ba4d857abc9fd4beade1d1e2e8b5f676bf84c518f2e01d2a4a836
Sep 11 16:49:31.293256 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:31.293218    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7780-72ebe9" event={"ID":"9d709ec6-2593-4929-a238-f95779370d3f","Type":"ContainerStarted","Data":"30840b1c3a4932608b503f03571ba6d8968eee2f5efdfa6661f9d1ca21edf533"}
Sep 11 16:49:31.293256 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:31.293263    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7780-72ebe9" event={"ID":"9d709ec6-2593-4929-a238-f95779370d3f","Type":"ContainerStarted","Data":"cb70b980184ba4d857abc9fd4beade1d1e2e8b5f676bf84c518f2e01d2a4a836"}
Sep 11 16:49:31.293713 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:31.293288    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7780-fc6d45" podUID="b5bd8d36-a276-4d3b-8109-b1a10c13465f" containerName="test-curl-7780-fc6d45" containerID="cri-o://b50bdf5306434a718d481705f0d983a95038281b1951574cd8b102043eaa4290" gracePeriod=2
Sep 11 16:49:31.296193 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:31.296169    2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7780-fc6d45\" is forbidden: User \"system:node:ip-10-0-135-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-58.ec2.internal' and this object" podUID="b5bd8d36-a276-4d3b-8109-b1a10c13465f" pod="opendatahub/test-curl-7780-fc6d45"
Sep 11 16:49:31.326934 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:31.326890    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7780-72ebe9" podStartSLOduration=1.326876907 podStartE2EDuration="1.326876907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:49:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:31.324895749 +0000 UTC m=+1189.878065685" watchObservedRunningTime="2026-09-11 16:49:31.326876907 +0000 UTC m=+1189.880046825"
Sep 11 16:49:31.327667 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:31.327643    2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7780-fc6d45\" is forbidden: User \"system:node:ip-10-0-135-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-58.ec2.internal' and this object" podUID="b5bd8d36-a276-4d3b-8109-b1a10c13465f" pod="opendatahub/test-curl-7780-fc6d45"
Sep 11 16:49:31.937671 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:31.937636    2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7780-fc6d45\" is forbidden: User \"system:node:ip-10-0-135-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-58.ec2.internal' and this object" podUID="b5bd8d36-a276-4d3b-8109-b1a10c13465f" pod="opendatahub/test-curl-7780-fc6d45"
Sep 11 16:49:32.040937 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:32.040839    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7780-72ebe9"]
Sep 11 16:49:32.043214 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:32.043188    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7780-72ebe9"]
Sep 11 16:49:32.162578 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:32.162549    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7780-e641ad"]
Sep 11 16:49:32.163016 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:32.163001    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d709ec6-2593-4929-a238-f95779370d3f" containerName="test-curl-7780-72ebe9"
Sep 11 16:49:32.163072 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:32.163018    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d709ec6-2593-4929-a238-f95779370d3f" containerName="test-curl-7780-72ebe9"
Sep 11 16:49:32.163108 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:32.163084    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d709ec6-2593-4929-a238-f95779370d3f" containerName="test-curl-7780-72ebe9"
Sep 11 16:49:32.166160 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:32.166146    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-e641ad"
Sep 11 16:49:32.174038 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:32.174019    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7780-e641ad"]
Sep 11 16:49:32.231948 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:32.231921    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ch4t\" (UniqueName: \"kubernetes.io/projected/17b455f8-22e2-44a9-b249-04e3bef3466b-kube-api-access-2ch4t\") pod \"test-curl-7780-e641ad\" (UID: \"17b455f8-22e2-44a9-b249-04e3bef3466b\") " pod="opendatahub/test-curl-7780-e641ad"
Sep 11 16:49:32.299161 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:32.299103    2545 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7780-72ebe9" container="test-curl-7780-72ebe9"
Sep 11 16:49:32.332999 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:32.332974    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ch4t\" (UniqueName: \"kubernetes.io/projected/17b455f8-22e2-44a9-b249-04e3bef3466b-kube-api-access-2ch4t\") pod \"test-curl-7780-e641ad\" (UID: \"17b455f8-22e2-44a9-b249-04e3bef3466b\") " pod="opendatahub/test-curl-7780-e641ad"
Sep 11 16:49:32.343676 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:32.343655    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ch4t\" (UniqueName: \"kubernetes.io/projected/17b455f8-22e2-44a9-b249-04e3bef3466b-kube-api-access-2ch4t\") pod \"test-curl-7780-e641ad\" (UID: \"17b455f8-22e2-44a9-b249-04e3bef3466b\") " pod="opendatahub/test-curl-7780-e641ad"
Sep 11 16:49:32.476211 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:32.476181    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-e641ad"
Sep 11 16:49:32.766042 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:32.766009    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7780-e641ad"]
Sep 11 16:49:32.769031 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:49:32.769000    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17b455f8_22e2_44a9_b249_04e3bef3466b.slice/crio-9be309d18ac6d1a29f19b295a525868b37d384fc836440f08226bdf57f6340e3 WatchSource:0}: Error finding container 9be309d18ac6d1a29f19b295a525868b37d384fc836440f08226bdf57f6340e3: Status 404 returned error can't find the container with id 9be309d18ac6d1a29f19b295a525868b37d384fc836440f08226bdf57f6340e3
Sep 11 16:49:33.309697 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:33.309665    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7780-e641ad" event={"ID":"17b455f8-22e2-44a9-b249-04e3bef3466b","Type":"ContainerStarted","Data":"5dc9b440cd692ffe45f1a3348e0b4a6a5cd5346beb2042a2ded29ed9f4154925"}
Sep 11 16:49:33.310034 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:33.309710    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7780-e641ad" event={"ID":"17b455f8-22e2-44a9-b249-04e3bef3466b","Type":"ContainerStarted","Data":"9be309d18ac6d1a29f19b295a525868b37d384fc836440f08226bdf57f6340e3"}
Sep 11 16:49:33.310034 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:33.309781    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7780-72ebe9" podUID="9d709ec6-2593-4929-a238-f95779370d3f" containerName="test-curl-7780-72ebe9" containerID="cri-o://30840b1c3a4932608b503f03571ba6d8968eee2f5efdfa6661f9d1ca21edf533" gracePeriod=2
Sep 11 16:49:33.337299 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:33.337252    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7780-e641ad" podStartSLOduration=1.3372373 podStartE2EDuration="1.3372373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:49:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:33.335274735 +0000 UTC m=+1191.888444661" watchObservedRunningTime="2026-09-11 16:49:33.3372373 +0000 UTC m=+1191.890407217"
Sep 11 16:49:33.337409 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:33.337331    2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7780-72ebe9\" is forbidden: User \"system:node:ip-10-0-135-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-58.ec2.internal' and this object" podUID="9d709ec6-2593-4929-a238-f95779370d3f" pod="opendatahub/test-curl-7780-72ebe9"
Sep 11 16:49:33.484273 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:33.484250    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-fc6d45"
Sep 11 16:49:33.489797 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:33.489770    2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7780-72ebe9\" is forbidden: User \"system:node:ip-10-0-135-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-58.ec2.internal' and this object" podUID="9d709ec6-2593-4929-a238-f95779370d3f" pod="opendatahub/test-curl-7780-72ebe9"
Sep 11 16:49:33.546265 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:33.546240    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lgh4\" (UniqueName: \"kubernetes.io/projected/b5bd8d36-a276-4d3b-8109-b1a10c13465f-kube-api-access-5lgh4\") pod \"b5bd8d36-a276-4d3b-8109-b1a10c13465f\" (UID: \"b5bd8d36-a276-4d3b-8109-b1a10c13465f\") "
Sep 11 16:49:33.548572 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:33.548508    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5bd8d36-a276-4d3b-8109-b1a10c13465f-kube-api-access-5lgh4" (OuterVolumeSpecName: "kube-api-access-5lgh4") pod "b5bd8d36-a276-4d3b-8109-b1a10c13465f" (UID: "b5bd8d36-a276-4d3b-8109-b1a10c13465f"). InnerVolumeSpecName "kube-api-access-5lgh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:49:33.647058 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:33.647030    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lgh4\" (UniqueName: \"kubernetes.io/projected/b5bd8d36-a276-4d3b-8109-b1a10c13465f-kube-api-access-5lgh4\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:49:33.938490 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:33.938416    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5bd8d36-a276-4d3b-8109-b1a10c13465f" path="/var/lib/kubelet/pods/b5bd8d36-a276-4d3b-8109-b1a10c13465f/volumes"
Sep 11 16:49:33.972525 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:33.972500    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7780-e641ad"]
Sep 11 16:49:33.978669 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:33.978648    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7780-e641ad"]
Sep 11 16:49:34.098890 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.098836    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7780-c16900"]
Sep 11 16:49:34.099321 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.099303    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17b455f8-22e2-44a9-b249-04e3bef3466b" containerName="test-curl-7780-e641ad"
Sep 11 16:49:34.099409 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.099324    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="17b455f8-22e2-44a9-b249-04e3bef3466b" containerName="test-curl-7780-e641ad"
Sep 11 16:49:34.099480 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.099431    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="17b455f8-22e2-44a9-b249-04e3bef3466b" containerName="test-curl-7780-e641ad"
Sep 11 16:49:34.102677 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.102659    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-c16900"
Sep 11 16:49:34.110388 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.110368    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7780-c16900"]
Sep 11 16:49:34.151720 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.151697    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2lgp\" (UniqueName: \"kubernetes.io/projected/95188c3f-7674-4c27-b886-f0dd05c2cda6-kube-api-access-h2lgp\") pod \"test-curl-7780-c16900\" (UID: \"95188c3f-7674-4c27-b886-f0dd05c2cda6\") " pod="opendatahub/test-curl-7780-c16900"
Sep 11 16:49:34.253112 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.253086    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2lgp\" (UniqueName: \"kubernetes.io/projected/95188c3f-7674-4c27-b886-f0dd05c2cda6-kube-api-access-h2lgp\") pod \"test-curl-7780-c16900\" (UID: \"95188c3f-7674-4c27-b886-f0dd05c2cda6\") " pod="opendatahub/test-curl-7780-c16900"
Sep 11 16:49:34.263596 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.263574    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2lgp\" (UniqueName: \"kubernetes.io/projected/95188c3f-7674-4c27-b886-f0dd05c2cda6-kube-api-access-h2lgp\") pod \"test-curl-7780-c16900\" (UID: \"95188c3f-7674-4c27-b886-f0dd05c2cda6\") " pod="opendatahub/test-curl-7780-c16900"
Sep 11 16:49:34.318007 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.317977    2545 generic.go:356] "Generic (PLEG): container finished" podID="b5bd8d36-a276-4d3b-8109-b1a10c13465f" containerID="b50bdf5306434a718d481705f0d983a95038281b1951574cd8b102043eaa4290" exitCode=137
Sep 11 16:49:34.318348 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.318015    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-fc6d45"
Sep 11 16:49:34.318348 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.318079    2545 scope.go:117] "RemoveContainer" containerID="b50bdf5306434a718d481705f0d983a95038281b1951574cd8b102043eaa4290"
Sep 11 16:49:34.318348 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:34.318313    2545 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7780-e641ad" container="test-curl-7780-e641ad"
Sep 11 16:49:34.326225 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.326204    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"]
Sep 11 16:49:34.327455 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.327435    2545 scope.go:117] "RemoveContainer" containerID="b50bdf5306434a718d481705f0d983a95038281b1951574cd8b102043eaa4290"
Sep 11 16:49:34.327717 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:34.327700    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b50bdf5306434a718d481705f0d983a95038281b1951574cd8b102043eaa4290\": container with ID starting with b50bdf5306434a718d481705f0d983a95038281b1951574cd8b102043eaa4290 not found: ID does not exist" containerID="b50bdf5306434a718d481705f0d983a95038281b1951574cd8b102043eaa4290"
Sep 11 16:49:34.327788 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.327726    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b50bdf5306434a718d481705f0d983a95038281b1951574cd8b102043eaa4290"} err="failed to get container status \"b50bdf5306434a718d481705f0d983a95038281b1951574cd8b102043eaa4290\": rpc error: code = NotFound desc = could not find container \"b50bdf5306434a718d481705f0d983a95038281b1951574cd8b102043eaa4290\": container with ID starting with b50bdf5306434a718d481705f0d983a95038281b1951574cd8b102043eaa4290 not found: ID does not exist"
Sep 11 16:49:34.331051 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.331032    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.333647 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.333625    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ff7416\"/\"default-dockercfg-2nm4h\""
Sep 11 16:49:34.334043 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.334025    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ff7416\"/\"kube-root-ca.crt\""
Sep 11 16:49:34.334295 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.334277    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ff7416\"/\"auth-test-model-ff7416-kserve-self-signed-certs\""
Sep 11 16:49:34.334341 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.334325    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ff7416\"/\"openshift-service-ca.crt\""
Sep 11 16:49:34.344820 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.344798    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"]
Sep 11 16:49:34.413063 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.413027    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-c16900"
Sep 11 16:49:34.455146 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.455115    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-tmp-dir\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.455298 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.455153    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7jtq\" (UniqueName: \"kubernetes.io/projected/fda59737-bb8a-48f8-baf1-6a203c120772-kube-api-access-v7jtq\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.455298 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.455210    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-dshm\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.455298 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.455274    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-home\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.455477 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.455306    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda59737-bb8a-48f8-baf1-6a203c120772-tls-certs\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.455477 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.455350    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-model-cache\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.555945 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.555844    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-tmp-dir\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.555945 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.555918    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7jtq\" (UniqueName: \"kubernetes.io/projected/fda59737-bb8a-48f8-baf1-6a203c120772-kube-api-access-v7jtq\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.556150 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.555969    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-dshm\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.556150 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.556022    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-home\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.556150 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.556053    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda59737-bb8a-48f8-baf1-6a203c120772-tls-certs\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.556150 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.556100    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-model-cache\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.556339 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.556301    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-tmp-dir\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.556402 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.556385    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-home\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.556456 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.556422    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-model-cache\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.558398 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.558379    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-dshm\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.558552 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.558536    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda59737-bb8a-48f8-baf1-6a203c120772-tls-certs\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.569845 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.569813    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7jtq\" (UniqueName: \"kubernetes.io/projected/fda59737-bb8a-48f8-baf1-6a203c120772-kube-api-access-v7jtq\") pod \"auth-test-model-ff7416-kserve-99b8cd4c-sfbmr\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.639942 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.639915    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:34.700262 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.700230    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7780-c16900"]
Sep 11 16:49:34.923186 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:34.923162    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"]
Sep 11 16:49:34.925613 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:49:34.925586    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfda59737_bb8a_48f8_baf1_6a203c120772.slice/crio-cddc38c78ed0c5cd7e9aa4606721755f11435e17b7e60dfa8844c486c13aa1f2 WatchSource:0}: Error finding container cddc38c78ed0c5cd7e9aa4606721755f11435e17b7e60dfa8844c486c13aa1f2: Status 404 returned error can't find the container with id cddc38c78ed0c5cd7e9aa4606721755f11435e17b7e60dfa8844c486c13aa1f2
Sep 11 16:49:35.326088 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:35.326052    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr" event={"ID":"fda59737-bb8a-48f8-baf1-6a203c120772","Type":"ContainerStarted","Data":"c063ddbc4acc441e5ca9335043ac138a8080c3927a9e5dd2f268ad31b66cebf2"}
Sep 11 16:49:35.326458 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:35.326104    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr" event={"ID":"fda59737-bb8a-48f8-baf1-6a203c120772","Type":"ContainerStarted","Data":"cddc38c78ed0c5cd7e9aa4606721755f11435e17b7e60dfa8844c486c13aa1f2"}
Sep 11 16:49:35.327529 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:35.327501    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7780-c16900" event={"ID":"95188c3f-7674-4c27-b886-f0dd05c2cda6","Type":"ContainerStarted","Data":"bfe539701b20b73129064711c5b3a0b63557ae9f8905b619adc1ecbe9c21e563"}
Sep 11 16:49:35.327646 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:35.327543    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7780-c16900" event={"ID":"95188c3f-7674-4c27-b886-f0dd05c2cda6","Type":"ContainerStarted","Data":"18b6cc5d46b31cf087ca91657393a77dd50ef82daa320b543167f91d571e0f43"}
Sep 11 16:49:35.327706 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:35.327644    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7780-e641ad" podUID="17b455f8-22e2-44a9-b249-04e3bef3466b" containerName="test-curl-7780-e641ad" containerID="cri-o://5dc9b440cd692ffe45f1a3348e0b4a6a5cd5346beb2042a2ded29ed9f4154925" gracePeriod=2
Sep 11 16:49:35.366824 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:35.366768    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr" podStartSLOduration=1.366748785 podStartE2EDuration="1.366748785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:49:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:35.364458488 +0000 UTC m=+1193.917628442" watchObservedRunningTime="2026-09-11 16:49:35.366748785 +0000 UTC m=+1193.919918800"
Sep 11 16:49:35.369038 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:35.369004    2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7780-e641ad\" is forbidden: User \"system:node:ip-10-0-135-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-58.ec2.internal' and this object" podUID="17b455f8-22e2-44a9-b249-04e3bef3466b" pod="opendatahub/test-curl-7780-e641ad"
Sep 11 16:49:35.505691 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:35.505671    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-72ebe9"
Sep 11 16:49:35.507938 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:35.507905    2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7780-e641ad\" is forbidden: User \"system:node:ip-10-0-135-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-58.ec2.internal' and this object" podUID="17b455f8-22e2-44a9-b249-04e3bef3466b" pod="opendatahub/test-curl-7780-e641ad"
Sep 11 16:49:35.565797 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:35.565749    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhx4g\" (UniqueName: \"kubernetes.io/projected/9d709ec6-2593-4929-a238-f95779370d3f-kube-api-access-hhx4g\") pod \"9d709ec6-2593-4929-a238-f95779370d3f\" (UID: \"9d709ec6-2593-4929-a238-f95779370d3f\") "
Sep 11 16:49:35.567928 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:35.567898    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d709ec6-2593-4929-a238-f95779370d3f-kube-api-access-hhx4g" (OuterVolumeSpecName: "kube-api-access-hhx4g") pod "9d709ec6-2593-4929-a238-f95779370d3f" (UID: "9d709ec6-2593-4929-a238-f95779370d3f"). InnerVolumeSpecName "kube-api-access-hhx4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:49:35.667064 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:35.667041    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhx4g\" (UniqueName: \"kubernetes.io/projected/9d709ec6-2593-4929-a238-f95779370d3f-kube-api-access-hhx4g\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:49:35.938096 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:35.938026    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d709ec6-2593-4929-a238-f95779370d3f" path="/var/lib/kubelet/pods/9d709ec6-2593-4929-a238-f95779370d3f/volumes"
Sep 11 16:49:36.031755 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:36.031705    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7780-c16900" podStartSLOduration=2.0316897 podStartE2EDuration="2.0316897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:49:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:35.404165155 +0000 UTC m=+1193.957335081" watchObservedRunningTime="2026-09-11 16:49:36.0316897 +0000 UTC m=+1194.584859683"
Sep 11 16:49:36.033353 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:36.033329    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7780-c16900"]
Sep 11 16:49:36.042568 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:36.042547    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7780-c16900"]
Sep 11 16:49:36.335478 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:36.335445    2545 generic.go:356] "Generic (PLEG): container finished" podID="9d709ec6-2593-4929-a238-f95779370d3f" containerID="30840b1c3a4932608b503f03571ba6d8968eee2f5efdfa6661f9d1ca21edf533" exitCode=137
Sep 11 16:49:36.335920 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:36.335484    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-72ebe9"
Sep 11 16:49:36.335920 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:36.335547    2545 scope.go:117] "RemoveContainer" containerID="30840b1c3a4932608b503f03571ba6d8968eee2f5efdfa6661f9d1ca21edf533"
Sep 11 16:49:36.336076 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:36.335946    2545 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7780-c16900" container="test-curl-7780-c16900"
Sep 11 16:49:36.344376 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:36.344358    2545 scope.go:117] "RemoveContainer" containerID="30840b1c3a4932608b503f03571ba6d8968eee2f5efdfa6661f9d1ca21edf533"
Sep 11 16:49:36.344652 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:36.344634    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30840b1c3a4932608b503f03571ba6d8968eee2f5efdfa6661f9d1ca21edf533\": container with ID starting with 30840b1c3a4932608b503f03571ba6d8968eee2f5efdfa6661f9d1ca21edf533 not found: ID does not exist" containerID="30840b1c3a4932608b503f03571ba6d8968eee2f5efdfa6661f9d1ca21edf533"
Sep 11 16:49:36.344716 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:36.344666    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30840b1c3a4932608b503f03571ba6d8968eee2f5efdfa6661f9d1ca21edf533"} err="failed to get container status \"30840b1c3a4932608b503f03571ba6d8968eee2f5efdfa6661f9d1ca21edf533\": rpc error: code = NotFound desc = could not find container \"30840b1c3a4932608b503f03571ba6d8968eee2f5efdfa6661f9d1ca21edf533\": container with ID starting with 30840b1c3a4932608b503f03571ba6d8968eee2f5efdfa6661f9d1ca21edf533 not found: ID does not exist"
Sep 11 16:49:36.980523 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:36.980488    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"]
Sep 11 16:49:36.980959 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:36.980945    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95188c3f-7674-4c27-b886-f0dd05c2cda6" containerName="test-curl-7780-c16900"
Sep 11 16:49:36.981024 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:36.980961    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="95188c3f-7674-4c27-b886-f0dd05c2cda6" containerName="test-curl-7780-c16900"
Sep 11 16:49:36.981059 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:36.981045    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="95188c3f-7674-4c27-b886-f0dd05c2cda6" containerName="test-curl-7780-c16900"
Sep 11 16:49:36.985630 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:36.985611    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:36.995534 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:36.995515    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3bf678c7-openshift-default-dockercfg-xgk9p\""
Sep 11 16:49:37.008634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.008614    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"]
Sep 11 16:49:37.080067 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.080019    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-credential-socket\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.080266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.080096    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c197f954-919c-467b-8d00-f15040f84c23-istiod-ca-cert\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.080266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.080169    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-istio-data\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.080266 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.080217    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-workload-certs\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.080726 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.080309    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-istio-envoy\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.080726 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.080353    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c197f954-919c-467b-8d00-f15040f84c23-istio-token\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.080726 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.080411    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nvpb\" (UniqueName: \"kubernetes.io/projected/c197f954-919c-467b-8d00-f15040f84c23-kube-api-access-2nvpb\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.080726 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.080518    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-workload-socket\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.080726 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.080564    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c197f954-919c-467b-8d00-f15040f84c23-istio-podinfo\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.181402 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.181366    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-istio-data\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.181558 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.181421    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-workload-certs\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.181558 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.181463    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-istio-envoy\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.181558 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.181490    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c197f954-919c-467b-8d00-f15040f84c23-istio-token\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.181558 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.181542    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nvpb\" (UniqueName: \"kubernetes.io/projected/c197f954-919c-467b-8d00-f15040f84c23-kube-api-access-2nvpb\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.181778 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.181604    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-workload-socket\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.181778 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.181665    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c197f954-919c-467b-8d00-f15040f84c23-istio-podinfo\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.181902 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.181820    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-credential-socket\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.181902 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.181827    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-istio-data\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.182012 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.181908    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c197f954-919c-467b-8d00-f15040f84c23-istiod-ca-cert\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.182335 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.182286    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-credential-socket\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.182335 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.182296    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-workload-socket\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.182335 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.182299    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-workload-certs\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.182851 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.182831    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c197f954-919c-467b-8d00-f15040f84c23-istiod-ca-cert\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.184305 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.184286    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-istio-envoy\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.184415 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.184397    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c197f954-919c-467b-8d00-f15040f84c23-istio-podinfo\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.191133 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.191111    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c197f954-919c-467b-8d00-f15040f84c23-istio-token\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.191325 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.191305    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nvpb\" (UniqueName: \"kubernetes.io/projected/c197f954-919c-467b-8d00-f15040f84c23-kube-api-access-2nvpb\") pod \"e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") " pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.297508 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.297475    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:37.344069 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.344041    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7780-c16900" podUID="95188c3f-7674-4c27-b886-f0dd05c2cda6" containerName="test-curl-7780-c16900" containerID="cri-o://bfe539701b20b73129064711c5b3a0b63557ae9f8905b619adc1ecbe9c21e563" gracePeriod=2
Sep 11 16:49:37.348075 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:37.348040    2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7780-c16900\" is forbidden: User \"system:node:ip-10-0-135-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-58.ec2.internal' and this object" podUID="95188c3f-7674-4c27-b886-f0dd05c2cda6" pod="opendatahub/test-curl-7780-c16900"
Sep 11 16:49:37.478246 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.478222    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-e641ad"
Sep 11 16:49:37.480177 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:37.480150    2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7780-c16900\" is forbidden: User \"system:node:ip-10-0-135-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-58.ec2.internal' and this object" podUID="95188c3f-7674-4c27-b886-f0dd05c2cda6" pod="opendatahub/test-curl-7780-c16900"
Sep 11 16:49:37.585403 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.585377    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ch4t\" (UniqueName: \"kubernetes.io/projected/17b455f8-22e2-44a9-b249-04e3bef3466b-kube-api-access-2ch4t\") pod \"17b455f8-22e2-44a9-b249-04e3bef3466b\" (UID: \"17b455f8-22e2-44a9-b249-04e3bef3466b\") "
Sep 11 16:49:37.585913 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.585885    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"]
Sep 11 16:49:37.587682 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:49:37.587658    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc197f954_919c_467b_8d00_f15040f84c23.slice/crio-bbd6637c465f69e37308562f433bf5c3d42556502f45bbf8e6ab8d55a77b73af WatchSource:0}: Error finding container bbd6637c465f69e37308562f433bf5c3d42556502f45bbf8e6ab8d55a77b73af: Status 404 returned error can't find the container with id bbd6637c465f69e37308562f433bf5c3d42556502f45bbf8e6ab8d55a77b73af
Sep 11 16:49:37.587783 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.587707    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17b455f8-22e2-44a9-b249-04e3bef3466b-kube-api-access-2ch4t" (OuterVolumeSpecName: "kube-api-access-2ch4t") pod "17b455f8-22e2-44a9-b249-04e3bef3466b" (UID: "17b455f8-22e2-44a9-b249-04e3bef3466b"). InnerVolumeSpecName "kube-api-access-2ch4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:49:37.590127 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.590092    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:49:37.590240 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.590160    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:49:37.590240 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.590190    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:49:37.686694 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.686652    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ch4t\" (UniqueName: \"kubernetes.io/projected/17b455f8-22e2-44a9-b249-04e3bef3466b-kube-api-access-2ch4t\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:49:37.939202 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:37.939127    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17b455f8-22e2-44a9-b249-04e3bef3466b" path="/var/lib/kubelet/pods/17b455f8-22e2-44a9-b249-04e3bef3466b/volumes"
Sep 11 16:49:38.351372 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:38.351335    2545 generic.go:356] "Generic (PLEG): container finished" podID="17b455f8-22e2-44a9-b249-04e3bef3466b" containerID="5dc9b440cd692ffe45f1a3348e0b4a6a5cd5346beb2042a2ded29ed9f4154925" exitCode=137
Sep 11 16:49:38.351847 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:38.351386    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-e641ad"
Sep 11 16:49:38.351847 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:38.351444    2545 scope.go:117] "RemoveContainer" containerID="5dc9b440cd692ffe45f1a3348e0b4a6a5cd5346beb2042a2ded29ed9f4154925"
Sep 11 16:49:38.353589 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:38.353563    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv" event={"ID":"c197f954-919c-467b-8d00-f15040f84c23","Type":"ContainerStarted","Data":"bcd533f6c13cc1a7f334241570c920638861177637fdc3fce9a44fda1c00ce0e"}
Sep 11 16:49:38.353716 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:38.353648    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv" event={"ID":"c197f954-919c-467b-8d00-f15040f84c23","Type":"ContainerStarted","Data":"bbd6637c465f69e37308562f433bf5c3d42556502f45bbf8e6ab8d55a77b73af"}
Sep 11 16:49:38.360857 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:38.360840    2545 scope.go:117] "RemoveContainer" containerID="5dc9b440cd692ffe45f1a3348e0b4a6a5cd5346beb2042a2ded29ed9f4154925"
Sep 11 16:49:38.361232 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:38.361214    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dc9b440cd692ffe45f1a3348e0b4a6a5cd5346beb2042a2ded29ed9f4154925\": container with ID starting with 5dc9b440cd692ffe45f1a3348e0b4a6a5cd5346beb2042a2ded29ed9f4154925 not found: ID does not exist" containerID="5dc9b440cd692ffe45f1a3348e0b4a6a5cd5346beb2042a2ded29ed9f4154925"
Sep 11 16:49:38.361310 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:38.361244    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dc9b440cd692ffe45f1a3348e0b4a6a5cd5346beb2042a2ded29ed9f4154925"} err="failed to get container status \"5dc9b440cd692ffe45f1a3348e0b4a6a5cd5346beb2042a2ded29ed9f4154925\": rpc error: code = NotFound desc = could not find container \"5dc9b440cd692ffe45f1a3348e0b4a6a5cd5346beb2042a2ded29ed9f4154925\": container with ID starting with 5dc9b440cd692ffe45f1a3348e0b4a6a5cd5346beb2042a2ded29ed9f4154925 not found: ID does not exist"
Sep 11 16:49:38.377582 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:38.377538    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv" podStartSLOduration=2.377524748 podStartE2EDuration="2.377524748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:49:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:38.376501468 +0000 UTC m=+1196.929671400" watchObservedRunningTime="2026-09-11 16:49:38.377524748 +0000 UTC m=+1196.930694675"
Sep 11 16:49:39.298667 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:39.298630    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:39.304241 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:39.304215    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:39.361011 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:39.360992    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:39.361747 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:39.361728    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:49:39.541644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:39.541611    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-c16900"
Sep 11 16:49:39.604039 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:39.603980    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2lgp\" (UniqueName: \"kubernetes.io/projected/95188c3f-7674-4c27-b886-f0dd05c2cda6-kube-api-access-h2lgp\") pod \"95188c3f-7674-4c27-b886-f0dd05c2cda6\" (UID: \"95188c3f-7674-4c27-b886-f0dd05c2cda6\") "
Sep 11 16:49:39.606213 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:39.606186    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95188c3f-7674-4c27-b886-f0dd05c2cda6-kube-api-access-h2lgp" (OuterVolumeSpecName: "kube-api-access-h2lgp") pod "95188c3f-7674-4c27-b886-f0dd05c2cda6" (UID: "95188c3f-7674-4c27-b886-f0dd05c2cda6"). InnerVolumeSpecName "kube-api-access-h2lgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:49:39.705360 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:39.705333    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2lgp\" (UniqueName: \"kubernetes.io/projected/95188c3f-7674-4c27-b886-f0dd05c2cda6-kube-api-access-h2lgp\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:49:39.937601 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:39.937532    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95188c3f-7674-4c27-b886-f0dd05c2cda6" path="/var/lib/kubelet/pods/95188c3f-7674-4c27-b886-f0dd05c2cda6/volumes"
Sep 11 16:49:40.078638 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:40.078553    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:49:40.368245 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:40.368212    2545 generic.go:356] "Generic (PLEG): container finished" podID="95188c3f-7674-4c27-b886-f0dd05c2cda6" containerID="bfe539701b20b73129064711c5b3a0b63557ae9f8905b619adc1ecbe9c21e563" exitCode=137
Sep 11 16:49:40.368245 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:40.368247    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7780-c16900"
Sep 11 16:49:40.368758 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:40.368301    2545 scope.go:117] "RemoveContainer" containerID="bfe539701b20b73129064711c5b3a0b63557ae9f8905b619adc1ecbe9c21e563"
Sep 11 16:49:40.378441 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:40.378421    2545 scope.go:117] "RemoveContainer" containerID="bfe539701b20b73129064711c5b3a0b63557ae9f8905b619adc1ecbe9c21e563"
Sep 11 16:49:40.378760 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:49:40.378735    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfe539701b20b73129064711c5b3a0b63557ae9f8905b619adc1ecbe9c21e563\": container with ID starting with bfe539701b20b73129064711c5b3a0b63557ae9f8905b619adc1ecbe9c21e563 not found: ID does not exist" containerID="bfe539701b20b73129064711c5b3a0b63557ae9f8905b619adc1ecbe9c21e563"
Sep 11 16:49:40.378896 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:40.378761    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfe539701b20b73129064711c5b3a0b63557ae9f8905b619adc1ecbe9c21e563"} err="failed to get container status \"bfe539701b20b73129064711c5b3a0b63557ae9f8905b619adc1ecbe9c21e563\": rpc error: code = NotFound desc = could not find container \"bfe539701b20b73129064711c5b3a0b63557ae9f8905b619adc1ecbe9c21e563\": container with ID starting with bfe539701b20b73129064711c5b3a0b63557ae9f8905b619adc1ecbe9c21e563 not found: ID does not exist"
Sep 11 16:49:41.991641 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:41.991608    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:49:41.994011 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:41.993990    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:49:42.115724 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:42.115694    2545 scope.go:117] "RemoveContainer" containerID="b6a6ff8db2390a1ae825fbde8d5c13e02b69dcee8bdb2bc94a466a4332063e68"
Sep 11 16:49:44.640920 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:44.640883    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:44.641282 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:44.640945    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:44.653617 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:44.653593    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:45.413735 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:45.413711    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:49:56.364749 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:56.364708    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:49:58.483878 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:58.483787    2545 generic.go:356] "Generic (PLEG): container finished" podID="c197f954-919c-467b-8d00-f15040f84c23" containerID="bcd533f6c13cc1a7f334241570c920638861177637fdc3fce9a44fda1c00ce0e" exitCode=0
Sep 11 16:49:58.483878 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:58.483837    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv" event={"ID":"c197f954-919c-467b-8d00-f15040f84c23","Type":"ContainerDied","Data":"bcd533f6c13cc1a7f334241570c920638861177637fdc3fce9a44fda1c00ce0e"}
Sep 11 16:49:58.484323 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:58.484251    2545 scope.go:117] "RemoveContainer" containerID="bcd533f6c13cc1a7f334241570c920638861177637fdc3fce9a44fda1c00ce0e"
Sep 11 16:49:58.485380 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:58.485352    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:49:58.485441 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:58.485424    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:49:58.485477 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:58.485456    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:49:59.496483 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:49:59.496433    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv" event={"ID":"c197f954-919c-467b-8d00-f15040f84c23","Type":"ContainerStarted","Data":"31ab4f2f05f907fd15b8f81a1583e5c5b497d12631a633f8494951b3a46b141f"}
Sep 11 16:50:00.298505 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.298465    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:50:00.298841 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.298811    2545 patch_prober.go:28] interesting pod/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 11 16:50:00.298964 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.298889    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv" podUID="c197f954-919c-467b-8d00-f15040f84c23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 11 16:50:00.509512 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.509479    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"]
Sep 11 16:50:00.513536 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.513511    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.516192 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.516170    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-ce902a\"/\"kube-root-ca.crt\""
Sep 11 16:50:00.516945 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.516922    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-ce902a\"/\"auth-test-model-ce902a-kserve-self-signed-certs\""
Sep 11 16:50:00.516945 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.516943    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-ce902a\"/\"openshift-service-ca.crt\""
Sep 11 16:50:00.517110 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.516943    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-ce902a\"/\"default-dockercfg-vj4dp\""
Sep 11 16:50:00.526466 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.526441    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"]
Sep 11 16:50:00.629893 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.629773    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-dshm\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.629893 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.629808    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r6ms\" (UniqueName: \"kubernetes.io/projected/915c120b-9044-45b1-88c1-b1c309ce5127-kube-api-access-8r6ms\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.630117 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.629898    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-tmp-dir\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.630117 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.629974    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/915c120b-9044-45b1-88c1-b1c309ce5127-tls-certs\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.630117 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.630062    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-model-cache\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.630117 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.630094    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-home\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.731686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.731642    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-dshm\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.731903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.731705    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r6ms\" (UniqueName: \"kubernetes.io/projected/915c120b-9044-45b1-88c1-b1c309ce5127-kube-api-access-8r6ms\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.731903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.731763    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-tmp-dir\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.731903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.731804    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/915c120b-9044-45b1-88c1-b1c309ce5127-tls-certs\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.732085 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.731908    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-model-cache\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.732085 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.731949    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-home\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.732212 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.732184    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-tmp-dir\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.732290 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.732268    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-model-cache\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.732349 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.732301    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-home\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.734912 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.734880    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-dshm\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.735417 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.735380    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/915c120b-9044-45b1-88c1-b1c309ce5127-tls-certs\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.744856 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.744828    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r6ms\" (UniqueName: \"kubernetes.io/projected/915c120b-9044-45b1-88c1-b1c309ce5127-kube-api-access-8r6ms\") pod \"auth-test-model-ce902a-kserve-c59957d7-vpf5f\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:00.824439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:00.824411    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:01.119784 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:01.119760    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"]
Sep 11 16:50:01.121812 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:50:01.121786    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod915c120b_9044_45b1_88c1_b1c309ce5127.slice/crio-c1384c218e9a04d51ab059675fe1cf9c22c80d9450e31aed64251d8ed3eba62e WatchSource:0}: Error finding container c1384c218e9a04d51ab059675fe1cf9c22c80d9450e31aed64251d8ed3eba62e: Status 404 returned error can't find the container with id c1384c218e9a04d51ab059675fe1cf9c22c80d9450e31aed64251d8ed3eba62e
Sep 11 16:50:01.298246 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:01.298201    2545 patch_prober.go:28] interesting pod/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 11 16:50:01.298440 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:01.298270    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv" podUID="c197f954-919c-467b-8d00-f15040f84c23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 11 16:50:01.516793 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:01.516762    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f" event={"ID":"915c120b-9044-45b1-88c1-b1c309ce5127","Type":"ContainerStarted","Data":"602d88daba5d56171a457de876529514a0ebfadd936a4242c37664177e2ec8b5"}
Sep 11 16:50:01.517237 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:01.516803    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f" event={"ID":"915c120b-9044-45b1-88c1-b1c309ce5127","Type":"ContainerStarted","Data":"c1384c218e9a04d51ab059675fe1cf9c22c80d9450e31aed64251d8ed3eba62e"}
Sep 11 16:50:01.558184 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:01.558139    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f" podStartSLOduration=1.55812516 podStartE2EDuration="1.55812516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:50:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:50:01.556331755 +0000 UTC m=+1220.109501702" watchObservedRunningTime="2026-09-11 16:50:01.55812516 +0000 UTC m=+1220.111295086"
Sep 11 16:50:02.298315 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:02.298268    2545 patch_prober.go:28] interesting pod/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 11 16:50:02.298538 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:02.298337    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv" podUID="c197f954-919c-467b-8d00-f15040f84c23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 11 16:50:03.298605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:03.298168    2545 patch_prober.go:28] interesting pod/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 11 16:50:03.298605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:03.298234    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv" podUID="c197f954-919c-467b-8d00-f15040f84c23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 11 16:50:04.298062 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:04.298028    2545 patch_prober.go:28] interesting pod/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 11 16:50:04.298247 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:04.298085    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv" podUID="c197f954-919c-467b-8d00-f15040f84c23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 11 16:50:05.302750 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:05.302717    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:50:05.303163 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:05.303086    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:50:05.303833 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:05.303816    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:50:10.825183 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:10.825144    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:10.825183 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:10.825189    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:10.837921 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:10.837897    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:11.598773 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:11.598745    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:50:14.727363 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.727330    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"]
Sep 11 16:50:14.730968 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.730950    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.733146 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.733125    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3bf678c7\"/\"openshift-service-ca.crt\""
Sep 11 16:50:14.734054 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.734031    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3bf678c7\"/\"e2e-lifecycle-model-3bf678c7-kserve-self-signed-certs\""
Sep 11 16:50:14.734147 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.734038    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3bf678c7\"/\"default-dockercfg-g4lhd\""
Sep 11 16:50:14.734690 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.734677    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3bf678c7\"/\"kube-root-ca.crt\""
Sep 11 16:50:14.747711 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.747691    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"]
Sep 11 16:50:14.867690 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.867661    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-tmp-dir\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.867846 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.867755    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-model-cache\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.867846 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.867802    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-home\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.867982 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.867850    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-dshm\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.867982 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.867897    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-tls-certs\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.867982 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.867928    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzhh4\" (UniqueName: \"kubernetes.io/projected/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-kube-api-access-mzhh4\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.969157 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.969113    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-model-cache\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.969425 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.969207    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-home\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.969425 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.969278    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-dshm\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.969425 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.969331    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-tls-certs\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.969616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.969597    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-model-cache\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.969778 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.969751    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzhh4\" (UniqueName: \"kubernetes.io/projected/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-kube-api-access-mzhh4\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.969942 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.969810    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-tmp-dir\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.969942 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.969898    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-home\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.970193 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.970174    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-tmp-dir\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.971807 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.971782    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-dshm\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.972421 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.972401    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-tls-certs\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:14.981081 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:14.981031    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzhh4\" (UniqueName: \"kubernetes.io/projected/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-kube-api-access-mzhh4\") pod \"e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") " pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:15.040526 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:15.040508    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:15.322689 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:15.322635    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"]
Sep 11 16:50:15.332505 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:50:15.332477    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd90db9a9_3ab8_4fd8_ad42_defcc0421bc5.slice/crio-028f3d84d953ddddb66656276c4d420f21f6b4b25e8950459dc74d5c57b24ced WatchSource:0}: Error finding container 028f3d84d953ddddb66656276c4d420f21f6b4b25e8950459dc74d5c57b24ced: Status 404 returned error can't find the container with id 028f3d84d953ddddb66656276c4d420f21f6b4b25e8950459dc74d5c57b24ced
Sep 11 16:50:15.364903 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:15.364876    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:50:15.612276 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:15.612187    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x" event={"ID":"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5","Type":"ContainerStarted","Data":"f52deac6c63075c2fba5624e12caf17557bf82aa9b2d9e1de8babb075797e0e2"}
Sep 11 16:50:15.612276 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:15.612235    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x" event={"ID":"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5","Type":"ContainerStarted","Data":"028f3d84d953ddddb66656276c4d420f21f6b4b25e8950459dc74d5c57b24ced"}
Sep 11 16:50:15.638134 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:15.638084    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x" podStartSLOduration=1.6380674160000002 podStartE2EDuration="1.638067416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:50:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:50:15.636974182 +0000 UTC m=+1234.190144129" watchObservedRunningTime="2026-09-11 16:50:15.638067416 +0000 UTC m=+1234.191237343"
Sep 11 16:50:25.041417 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:25.041384    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:25.041911 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:25.041435    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:25.054089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:25.054065    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:25.685080 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:25.685049    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:50:33.060929 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:33.060893    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:50:39.921529 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:39.921496    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh"]
Sep 11 16:50:39.924910 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:39.924895    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh"
Sep 11 16:50:39.927286 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:39.927267    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:50:39.927400 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:39.927383    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:50:39.943522 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:39.943502    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh"]
Sep 11 16:50:39.999842 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:39.999796    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5e71421-6a0e-4764-ab79-2f33a599a384-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh\" (UID: \"d5e71421-6a0e-4764-ab79-2f33a599a384\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh"
Sep 11 16:50:40.101244 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:40.101218    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5e71421-6a0e-4764-ab79-2f33a599a384-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh\" (UID: \"d5e71421-6a0e-4764-ab79-2f33a599a384\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh"
Sep 11 16:50:40.101752 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:40.101733    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5e71421-6a0e-4764-ab79-2f33a599a384-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh\" (UID: \"d5e71421-6a0e-4764-ab79-2f33a599a384\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh"
Sep 11 16:50:40.235008 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:40.234941    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh"
Sep 11 16:50:40.516400 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:40.516376    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh"]
Sep 11 16:50:40.518687 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:50:40.518663    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5e71421_6a0e_4764_ab79_2f33a599a384.slice/crio-f8bb9b146aef6eafdee4424e207b2b14ca10505a93dd1f17e8994c352f63f4fa WatchSource:0}: Error finding container f8bb9b146aef6eafdee4424e207b2b14ca10505a93dd1f17e8994c352f63f4fa: Status 404 returned error can't find the container with id f8bb9b146aef6eafdee4424e207b2b14ca10505a93dd1f17e8994c352f63f4fa
Sep 11 16:50:40.765880 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:40.765797    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh" event={"ID":"d5e71421-6a0e-4764-ab79-2f33a599a384","Type":"ContainerStarted","Data":"68cc8e54acc1e46660d7e9b463073af28caa0023e16e4707200fae6e7cb789eb"}
Sep 11 16:50:40.765880 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:40.765849    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh" event={"ID":"d5e71421-6a0e-4764-ab79-2f33a599a384","Type":"ContainerStarted","Data":"f8bb9b146aef6eafdee4424e207b2b14ca10505a93dd1f17e8994c352f63f4fa"}
Sep 11 16:50:40.793908 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:40.793840    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh" podStartSLOduration=1.793818186 podStartE2EDuration="1.793818186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:50:40.791979355 +0000 UTC m=+1259.345149280" watchObservedRunningTime="2026-09-11 16:50:40.793818186 +0000 UTC m=+1259.346988111"
Sep 11 16:50:41.776662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:41.776623    2545 generic.go:356] "Generic (PLEG): container finished" podID="d5e71421-6a0e-4764-ab79-2f33a599a384" containerID="68cc8e54acc1e46660d7e9b463073af28caa0023e16e4707200fae6e7cb789eb" exitCode=0
Sep 11 16:50:41.777165 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:41.776712    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh" event={"ID":"d5e71421-6a0e-4764-ab79-2f33a599a384","Type":"ContainerDied","Data":"68cc8e54acc1e46660d7e9b463073af28caa0023e16e4707200fae6e7cb789eb"}
Sep 11 16:50:42.159716 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:42.159639    2545 scope.go:117] "RemoveContainer" containerID="fc638e23644714330fcb6901bf80707e31a04aacc9414a7b74aa0e3c58988ee7"
Sep 11 16:50:42.168786 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:42.168762    2545 scope.go:117] "RemoveContainer" containerID="b928e64eaf00153edfccd7a89cd2960d084f1bcf6c56161df89ed343acc3e18c"
Sep 11 16:50:42.908250 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:42.908220    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh"
Sep 11 16:50:43.030473 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:43.030446    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5e71421-6a0e-4764-ab79-2f33a599a384-maas-api-service-ca\") pod \"d5e71421-6a0e-4764-ab79-2f33a599a384\" (UID: \"d5e71421-6a0e-4764-ab79-2f33a599a384\") "
Sep 11 16:50:43.030769 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:43.030746    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5e71421-6a0e-4764-ab79-2f33a599a384-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d5e71421-6a0e-4764-ab79-2f33a599a384" (UID: "d5e71421-6a0e-4764-ab79-2f33a599a384"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:50:43.132200 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:43.132132    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5e71421-6a0e-4764-ab79-2f33a599a384-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:50:43.790183 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:43.790153    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh"
Sep 11 16:50:43.790183 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:43.790184    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh" event={"ID":"d5e71421-6a0e-4764-ab79-2f33a599a384","Type":"ContainerDied","Data":"f8bb9b146aef6eafdee4424e207b2b14ca10505a93dd1f17e8994c352f63f4fa"}
Sep 11 16:50:43.790405 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:43.790228    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f8bb9b146aef6eafdee4424e207b2b14ca10505a93dd1f17e8994c352f63f4fa"
Sep 11 16:50:51.446603 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:51.446564    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:50:55.837259 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:55.837216    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh"]
Sep 11 16:50:55.840626 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:55.840599    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bf678c7-cwrjh"]
Sep 11 16:50:55.938640 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:50:55.938609    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5e71421-6a0e-4764-ab79-2f33a599a384" path="/var/lib/kubelet/pods/d5e71421-6a0e-4764-ab79-2f33a599a384/volumes"
Sep 11 16:51:04.037408 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:04.037375    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"]
Sep 11 16:51:09.351101 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:09.351065    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:51:19.056973 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.056918    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x" podUID="d90db9a9-3ab8-4fd8-ad42-defcc0421bc5" containerName="main" containerID="cri-o://f52deac6c63075c2fba5624e12caf17557bf82aa9b2d9e1de8babb075797e0e2" gracePeriod=45
Sep 11 16:51:19.323025 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.322968    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:51:19.365859 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.365819    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-tmp-dir\") pod \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") "
Sep 11 16:51:19.366080 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.365893    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzhh4\" (UniqueName: \"kubernetes.io/projected/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-kube-api-access-mzhh4\") pod \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") "
Sep 11 16:51:19.366080 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.365972    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-home\") pod \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") "
Sep 11 16:51:19.366080 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.366037    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-model-cache\") pod \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") "
Sep 11 16:51:19.366255 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.366135    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d90db9a9-3ab8-4fd8-ad42-defcc0421bc5" (UID: "d90db9a9-3ab8-4fd8-ad42-defcc0421bc5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:51:19.366255 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.366147    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-dshm\") pod \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") "
Sep 11 16:51:19.366255 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.366180    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-tls-certs\") pod \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\" (UID: \"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5\") "
Sep 11 16:51:19.366485 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.366323    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-model-cache" (OuterVolumeSpecName: "model-cache") pod "d90db9a9-3ab8-4fd8-ad42-defcc0421bc5" (UID: "d90db9a9-3ab8-4fd8-ad42-defcc0421bc5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:51:19.366545 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.366481    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-home" (OuterVolumeSpecName: "home") pod "d90db9a9-3ab8-4fd8-ad42-defcc0421bc5" (UID: "d90db9a9-3ab8-4fd8-ad42-defcc0421bc5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:51:19.366600 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.366549    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:19.366600 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.366567    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:19.368762 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.368727    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-kube-api-access-mzhh4" (OuterVolumeSpecName: "kube-api-access-mzhh4") pod "d90db9a9-3ab8-4fd8-ad42-defcc0421bc5" (UID: "d90db9a9-3ab8-4fd8-ad42-defcc0421bc5"). InnerVolumeSpecName "kube-api-access-mzhh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:51:19.368914 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.368794    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d90db9a9-3ab8-4fd8-ad42-defcc0421bc5" (UID: "d90db9a9-3ab8-4fd8-ad42-defcc0421bc5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:51:19.368997 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.368976    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-dshm" (OuterVolumeSpecName: "dshm") pod "d90db9a9-3ab8-4fd8-ad42-defcc0421bc5" (UID: "d90db9a9-3ab8-4fd8-ad42-defcc0421bc5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:51:19.467757 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.467723    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:19.467757 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.467756    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:19.467757 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.467767    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzhh4\" (UniqueName: \"kubernetes.io/projected/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-kube-api-access-mzhh4\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:19.468051 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.467775    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:19.998985 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.998949    2545 generic.go:356] "Generic (PLEG): container finished" podID="d90db9a9-3ab8-4fd8-ad42-defcc0421bc5" containerID="f52deac6c63075c2fba5624e12caf17557bf82aa9b2d9e1de8babb075797e0e2" exitCode=0
Sep 11 16:51:19.999139 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.999003    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"
Sep 11 16:51:19.999139 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.999011    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x" event={"ID":"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5","Type":"ContainerDied","Data":"f52deac6c63075c2fba5624e12caf17557bf82aa9b2d9e1de8babb075797e0e2"}
Sep 11 16:51:19.999139 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.999051    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x" event={"ID":"d90db9a9-3ab8-4fd8-ad42-defcc0421bc5","Type":"ContainerDied","Data":"028f3d84d953ddddb66656276c4d420f21f6b4b25e8950459dc74d5c57b24ced"}
Sep 11 16:51:19.999139 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:19.999074    2545 scope.go:117] "RemoveContainer" containerID="f52deac6c63075c2fba5624e12caf17557bf82aa9b2d9e1de8babb075797e0e2"
Sep 11 16:51:20.011730 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:20.011706    2545 scope.go:117] "RemoveContainer" containerID="f52deac6c63075c2fba5624e12caf17557bf82aa9b2d9e1de8babb075797e0e2"
Sep 11 16:51:20.012087 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:51:20.012067    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f52deac6c63075c2fba5624e12caf17557bf82aa9b2d9e1de8babb075797e0e2\": container with ID starting with f52deac6c63075c2fba5624e12caf17557bf82aa9b2d9e1de8babb075797e0e2 not found: ID does not exist" containerID="f52deac6c63075c2fba5624e12caf17557bf82aa9b2d9e1de8babb075797e0e2"
Sep 11 16:51:20.012169 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:20.012100    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f52deac6c63075c2fba5624e12caf17557bf82aa9b2d9e1de8babb075797e0e2"} err="failed to get container status \"f52deac6c63075c2fba5624e12caf17557bf82aa9b2d9e1de8babb075797e0e2\": rpc error: code = NotFound desc = could not find container \"f52deac6c63075c2fba5624e12caf17557bf82aa9b2d9e1de8babb075797e0e2\": container with ID starting with f52deac6c63075c2fba5624e12caf17557bf82aa9b2d9e1de8babb075797e0e2 not found: ID does not exist"
Sep 11 16:51:20.024506 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:20.024482    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"]
Sep 11 16:51:20.030809 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:20.030790    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-3bf678c7/e2e-lifecycle-model-3bf678c7-kserve-5ffbb95667-6sv6x"]
Sep 11 16:51:21.939842 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:21.939804    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d90db9a9-3ab8-4fd8-ad42-defcc0421bc5" path="/var/lib/kubelet/pods/d90db9a9-3ab8-4fd8-ad42-defcc0421bc5/volumes"
Sep 11 16:51:25.854322 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:25.854282    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:51:41.784587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:41.784542    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:51:42.195276 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:42.195201    2545 scope.go:117] "RemoveContainer" containerID="d43ddee46264aea3935e4d1126eca43a68fc0b4227b12218f96f5fcda5c80cf9"
Sep 11 16:51:42.204263 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:42.204244    2545 scope.go:117] "RemoveContainer" containerID="3e9b6e4733b01f270e9fd6ebe96b4b105eb513c61a8ebcef8bdf8578d7ced593"
Sep 11 16:51:42.211977 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:42.211955    2545 scope.go:117] "RemoveContainer" containerID="fc624d7509ea062ab2b8ea46cc791dee28d638850b057619a187cc8b2eed488b"
Sep 11 16:51:42.219559 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:42.219543    2545 scope.go:117] "RemoveContainer" containerID="e8f93196328682d272330465040da7a508aa861643555e53f25bc8c3e23a8db9"
Sep 11 16:51:42.227212 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:42.227195    2545 scope.go:117] "RemoveContainer" containerID="a747f976bb3aba0c7a5c2877fb820122f866ff2f5ce3b6020200cd194a9720b1"
Sep 11 16:51:49.181616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:49.181581    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"]
Sep 11 16:51:49.182070 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:49.181781    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv" podUID="c197f954-919c-467b-8d00-f15040f84c23" containerName="istio-proxy" containerID="cri-o://31ab4f2f05f907fd15b8f81a1583e5c5b497d12631a633f8494951b3a46b141f" gracePeriod=30
Sep 11 16:51:50.305338 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:50.305304    2545 patch_prober.go:28] interesting pod/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 16:51:50.305771 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:50.305373    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv" podUID="c197f954-919c-467b-8d00-f15040f84c23" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 16:51:54.451794 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.451738    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:51:54.501698 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.501671    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c197f954-919c-467b-8d00-f15040f84c23-istiod-ca-cert\") pod \"c197f954-919c-467b-8d00-f15040f84c23\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") "
Sep 11 16:51:54.501912 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.501706    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c197f954-919c-467b-8d00-f15040f84c23-istio-token\") pod \"c197f954-919c-467b-8d00-f15040f84c23\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") "
Sep 11 16:51:54.501912 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.501746    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nvpb\" (UniqueName: \"kubernetes.io/projected/c197f954-919c-467b-8d00-f15040f84c23-kube-api-access-2nvpb\") pod \"c197f954-919c-467b-8d00-f15040f84c23\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") "
Sep 11 16:51:54.501912 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.501780    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-istio-envoy\") pod \"c197f954-919c-467b-8d00-f15040f84c23\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") "
Sep 11 16:51:54.501912 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.501844    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-istio-data\") pod \"c197f954-919c-467b-8d00-f15040f84c23\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") "
Sep 11 16:51:54.501912 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.501884    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-workload-certs\") pod \"c197f954-919c-467b-8d00-f15040f84c23\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") "
Sep 11 16:51:54.502185 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.501924    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-credential-socket\") pod \"c197f954-919c-467b-8d00-f15040f84c23\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") "
Sep 11 16:51:54.502185 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.501963    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-workload-socket\") pod \"c197f954-919c-467b-8d00-f15040f84c23\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") "
Sep 11 16:51:54.502185 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.502077    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c197f954-919c-467b-8d00-f15040f84c23-istio-podinfo\") pod \"c197f954-919c-467b-8d00-f15040f84c23\" (UID: \"c197f954-919c-467b-8d00-f15040f84c23\") "
Sep 11 16:51:54.502185 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.502162    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-istio-data" (OuterVolumeSpecName: "istio-data") pod "c197f954-919c-467b-8d00-f15040f84c23" (UID: "c197f954-919c-467b-8d00-f15040f84c23"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:51:54.502372 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.502184    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c197f954-919c-467b-8d00-f15040f84c23-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c197f954-919c-467b-8d00-f15040f84c23" (UID: "c197f954-919c-467b-8d00-f15040f84c23"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:51:54.502372 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.502185    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c197f954-919c-467b-8d00-f15040f84c23" (UID: "c197f954-919c-467b-8d00-f15040f84c23"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:51:54.502372 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.502359    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c197f954-919c-467b-8d00-f15040f84c23" (UID: "c197f954-919c-467b-8d00-f15040f84c23"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:51:54.502510 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.502373    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c197f954-919c-467b-8d00-f15040f84c23" (UID: "c197f954-919c-467b-8d00-f15040f84c23"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:51:54.502510 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.502504    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c197f954-919c-467b-8d00-f15040f84c23-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:54.502589 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.502522    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:54.502589 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.502536    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:54.502589 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.502549    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:54.502701 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.502610    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:54.504201 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.504177    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c197f954-919c-467b-8d00-f15040f84c23-kube-api-access-2nvpb" (OuterVolumeSpecName: "kube-api-access-2nvpb") pod "c197f954-919c-467b-8d00-f15040f84c23" (UID: "c197f954-919c-467b-8d00-f15040f84c23"). InnerVolumeSpecName "kube-api-access-2nvpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:51:54.504319 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.504278    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c197f954-919c-467b-8d00-f15040f84c23-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c197f954-919c-467b-8d00-f15040f84c23" (UID: "c197f954-919c-467b-8d00-f15040f84c23"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:51:54.504640 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.504621    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c197f954-919c-467b-8d00-f15040f84c23" (UID: "c197f954-919c-467b-8d00-f15040f84c23"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:51:54.504684 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.504648    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c197f954-919c-467b-8d00-f15040f84c23-istio-token" (OuterVolumeSpecName: "istio-token") pod "c197f954-919c-467b-8d00-f15040f84c23" (UID: "c197f954-919c-467b-8d00-f15040f84c23"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:51:54.603399 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.603373    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c197f954-919c-467b-8d00-f15040f84c23-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:54.603399 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.603398    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c197f954-919c-467b-8d00-f15040f84c23-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:54.603554 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.603407    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nvpb\" (UniqueName: \"kubernetes.io/projected/c197f954-919c-467b-8d00-f15040f84c23-kube-api-access-2nvpb\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:54.603554 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:54.603416    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c197f954-919c-467b-8d00-f15040f84c23-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:51:55.210763 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:55.210733    2545 generic.go:356] "Generic (PLEG): container finished" podID="c197f954-919c-467b-8d00-f15040f84c23" containerID="31ab4f2f05f907fd15b8f81a1583e5c5b497d12631a633f8494951b3a46b141f" exitCode=0
Sep 11 16:51:55.210959 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:55.210771    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv" event={"ID":"c197f954-919c-467b-8d00-f15040f84c23","Type":"ContainerDied","Data":"31ab4f2f05f907fd15b8f81a1583e5c5b497d12631a633f8494951b3a46b141f"}
Sep 11 16:51:55.210959 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:55.210785    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"
Sep 11 16:51:55.210959 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:55.210818    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv" event={"ID":"c197f954-919c-467b-8d00-f15040f84c23","Type":"ContainerDied","Data":"bbd6637c465f69e37308562f433bf5c3d42556502f45bbf8e6ab8d55a77b73af"}
Sep 11 16:51:55.210959 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:55.210844    2545 scope.go:117] "RemoveContainer" containerID="31ab4f2f05f907fd15b8f81a1583e5c5b497d12631a633f8494951b3a46b141f"
Sep 11 16:51:55.220829 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:55.220812    2545 scope.go:117] "RemoveContainer" containerID="bcd533f6c13cc1a7f334241570c920638861177637fdc3fce9a44fda1c00ce0e"
Sep 11 16:51:55.229163 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:55.229144    2545 scope.go:117] "RemoveContainer" containerID="31ab4f2f05f907fd15b8f81a1583e5c5b497d12631a633f8494951b3a46b141f"
Sep 11 16:51:55.229426 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:51:55.229406    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31ab4f2f05f907fd15b8f81a1583e5c5b497d12631a633f8494951b3a46b141f\": container with ID starting with 31ab4f2f05f907fd15b8f81a1583e5c5b497d12631a633f8494951b3a46b141f not found: ID does not exist" containerID="31ab4f2f05f907fd15b8f81a1583e5c5b497d12631a633f8494951b3a46b141f"
Sep 11 16:51:55.229496 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:55.229439    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31ab4f2f05f907fd15b8f81a1583e5c5b497d12631a633f8494951b3a46b141f"} err="failed to get container status \"31ab4f2f05f907fd15b8f81a1583e5c5b497d12631a633f8494951b3a46b141f\": rpc error: code = NotFound desc = could not find container \"31ab4f2f05f907fd15b8f81a1583e5c5b497d12631a633f8494951b3a46b141f\": container with ID starting with 31ab4f2f05f907fd15b8f81a1583e5c5b497d12631a633f8494951b3a46b141f not found: ID does not exist"
Sep 11 16:51:55.229496 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:55.229482    2545 scope.go:117] "RemoveContainer" containerID="bcd533f6c13cc1a7f334241570c920638861177637fdc3fce9a44fda1c00ce0e"
Sep 11 16:51:55.229760 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:51:55.229738    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcd533f6c13cc1a7f334241570c920638861177637fdc3fce9a44fda1c00ce0e\": container with ID starting with bcd533f6c13cc1a7f334241570c920638861177637fdc3fce9a44fda1c00ce0e not found: ID does not exist" containerID="bcd533f6c13cc1a7f334241570c920638861177637fdc3fce9a44fda1c00ce0e"
Sep 11 16:51:55.229811 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:55.229766    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd533f6c13cc1a7f334241570c920638861177637fdc3fce9a44fda1c00ce0e"} err="failed to get container status \"bcd533f6c13cc1a7f334241570c920638861177637fdc3fce9a44fda1c00ce0e\": rpc error: code = NotFound desc = could not find container \"bcd533f6c13cc1a7f334241570c920638861177637fdc3fce9a44fda1c00ce0e\": container with ID starting with bcd533f6c13cc1a7f334241570c920638861177637fdc3fce9a44fda1c00ce0e not found: ID does not exist"
Sep 11 16:51:55.234620 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:55.234601    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"]
Sep 11 16:51:55.240038 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:55.240018    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3bf678c7-openshift-default-59c986bc8d-j2wcv"]
Sep 11 16:51:55.937636 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:55.937603    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c197f954-919c-467b-8d00-f15040f84c23" path="/var/lib/kubelet/pods/c197f954-919c-467b-8d00-f15040f84c23/volumes"
Sep 11 16:51:57.744873 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:57.744827    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:51:59.985331 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.985298    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7783-f665b1"]
Sep 11 16:51:59.985715 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.985702    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c197f954-919c-467b-8d00-f15040f84c23" containerName="istio-proxy"
Sep 11 16:51:59.985763 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.985716    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="c197f954-919c-467b-8d00-f15040f84c23" containerName="istio-proxy"
Sep 11 16:51:59.985763 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.985727    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c197f954-919c-467b-8d00-f15040f84c23" containerName="istio-proxy"
Sep 11 16:51:59.985763 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.985732    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="c197f954-919c-467b-8d00-f15040f84c23" containerName="istio-proxy"
Sep 11 16:51:59.985763 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.985756    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d90db9a9-3ab8-4fd8-ad42-defcc0421bc5" containerName="main"
Sep 11 16:51:59.985763 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.985761    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="d90db9a9-3ab8-4fd8-ad42-defcc0421bc5" containerName="main"
Sep 11 16:51:59.985953 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.985772    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5e71421-6a0e-4764-ab79-2f33a599a384" containerName="revoke-keys"
Sep 11 16:51:59.985953 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.985777    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5e71421-6a0e-4764-ab79-2f33a599a384" containerName="revoke-keys"
Sep 11 16:51:59.985953 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.985846    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5e71421-6a0e-4764-ab79-2f33a599a384" containerName="revoke-keys"
Sep 11 16:51:59.985953 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.985858    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="d90db9a9-3ab8-4fd8-ad42-defcc0421bc5" containerName="main"
Sep 11 16:51:59.985953 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.985879    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="c197f954-919c-467b-8d00-f15040f84c23" containerName="istio-proxy"
Sep 11 16:51:59.985953 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.985885    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="c197f954-919c-467b-8d00-f15040f84c23" containerName="istio-proxy"
Sep 11 16:51:59.990338 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.990320    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7783-f665b1"
Sep 11 16:51:59.992567 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.992545    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vtvd2\""
Sep 11 16:51:59.995996 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:51:59.995971    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7783-f665b1"]
Sep 11 16:52:00.052574 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:00.052532    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcsd8\" (UniqueName: \"kubernetes.io/projected/d5e18506-dbdb-45c0-8f77-a415da12a68a-kube-api-access-lcsd8\") pod \"test-curl-7783-f665b1\" (UID: \"d5e18506-dbdb-45c0-8f77-a415da12a68a\") " pod="opendatahub/test-curl-7783-f665b1"
Sep 11 16:52:00.153812 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:00.153770    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcsd8\" (UniqueName: \"kubernetes.io/projected/d5e18506-dbdb-45c0-8f77-a415da12a68a-kube-api-access-lcsd8\") pod \"test-curl-7783-f665b1\" (UID: \"d5e18506-dbdb-45c0-8f77-a415da12a68a\") " pod="opendatahub/test-curl-7783-f665b1"
Sep 11 16:52:00.165231 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:00.165202    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcsd8\" (UniqueName: \"kubernetes.io/projected/d5e18506-dbdb-45c0-8f77-a415da12a68a-kube-api-access-lcsd8\") pod \"test-curl-7783-f665b1\" (UID: \"d5e18506-dbdb-45c0-8f77-a415da12a68a\") " pod="opendatahub/test-curl-7783-f665b1"
Sep 11 16:52:00.301594 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:00.301561    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7783-f665b1"
Sep 11 16:52:00.582614 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:00.582588    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7783-f665b1"]
Sep 11 16:52:00.585314 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:52:00.585283    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5e18506_dbdb_45c0_8f77_a415da12a68a.slice/crio-66448cd524a098fe4eea333c70d92d4136f52a199520fb5e76d2236f50939982 WatchSource:0}: Error finding container 66448cd524a098fe4eea333c70d92d4136f52a199520fb5e76d2236f50939982: Status 404 returned error can't find the container with id 66448cd524a098fe4eea333c70d92d4136f52a199520fb5e76d2236f50939982
Sep 11 16:52:01.249186 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:01.249152    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7783-f665b1" event={"ID":"d5e18506-dbdb-45c0-8f77-a415da12a68a","Type":"ContainerStarted","Data":"d0fe2b8713165ec21aff402ac8185cb1d7cd2c7cc20bca918baf7f5f861a4f10"}
Sep 11 16:52:01.249630 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:01.249196    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7783-f665b1" event={"ID":"d5e18506-dbdb-45c0-8f77-a415da12a68a","Type":"ContainerStarted","Data":"66448cd524a098fe4eea333c70d92d4136f52a199520fb5e76d2236f50939982"}
Sep 11 16:52:01.267819 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:01.267770    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7783-f665b1" podStartSLOduration=2.267755451 podStartE2EDuration="2.267755451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:51:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:52:01.265699719 +0000 UTC m=+1339.818869647" watchObservedRunningTime="2026-09-11 16:52:01.267755451 +0000 UTC m=+1339.820925378"
Sep 11 16:52:01.912776 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:01.912743    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7783-f665b1"]
Sep 11 16:52:01.918571 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:01.918535    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7783-f665b1"]
Sep 11 16:52:02.254770 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:52:02.254723    2545 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7783-f665b1" container="test-curl-7783-f665b1"
Sep 11 16:52:03.260793 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:03.260751    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7783-f665b1" podUID="d5e18506-dbdb-45c0-8f77-a415da12a68a" containerName="test-curl-7783-f665b1" containerID="cri-o://d0fe2b8713165ec21aff402ac8185cb1d7cd2c7cc20bca918baf7f5f861a4f10" gracePeriod=2
Sep 11 16:52:03.435425 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:03.435345    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"]
Sep 11 16:52:04.408007 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:04.407977    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"]
Sep 11 16:52:05.448486 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:05.448450    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7783-f665b1"
Sep 11 16:52:05.504606 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:05.504570    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcsd8\" (UniqueName: \"kubernetes.io/projected/d5e18506-dbdb-45c0-8f77-a415da12a68a-kube-api-access-lcsd8\") pod \"d5e18506-dbdb-45c0-8f77-a415da12a68a\" (UID: \"d5e18506-dbdb-45c0-8f77-a415da12a68a\") "
Sep 11 16:52:05.506790 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:05.506753    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5e18506-dbdb-45c0-8f77-a415da12a68a-kube-api-access-lcsd8" (OuterVolumeSpecName: "kube-api-access-lcsd8") pod "d5e18506-dbdb-45c0-8f77-a415da12a68a" (UID: "d5e18506-dbdb-45c0-8f77-a415da12a68a"). InnerVolumeSpecName "kube-api-access-lcsd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:52:05.606068 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:05.605976    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcsd8\" (UniqueName: \"kubernetes.io/projected/d5e18506-dbdb-45c0-8f77-a415da12a68a-kube-api-access-lcsd8\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:52:05.938961 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:05.938849    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5e18506-dbdb-45c0-8f77-a415da12a68a" path="/var/lib/kubelet/pods/d5e18506-dbdb-45c0-8f77-a415da12a68a/volumes"
Sep 11 16:52:06.278958 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:06.278913    2545 generic.go:356] "Generic (PLEG): container finished" podID="d5e18506-dbdb-45c0-8f77-a415da12a68a" containerID="d0fe2b8713165ec21aff402ac8185cb1d7cd2c7cc20bca918baf7f5f861a4f10" exitCode=137
Sep 11 16:52:06.279169 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:06.278960    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7783-f665b1"
Sep 11 16:52:06.279169 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:06.278996    2545 scope.go:117] "RemoveContainer" containerID="d0fe2b8713165ec21aff402ac8185cb1d7cd2c7cc20bca918baf7f5f861a4f10"
Sep 11 16:52:06.288273 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:06.288254    2545 scope.go:117] "RemoveContainer" containerID="d0fe2b8713165ec21aff402ac8185cb1d7cd2c7cc20bca918baf7f5f861a4f10"
Sep 11 16:52:06.288649 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:52:06.288631    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0fe2b8713165ec21aff402ac8185cb1d7cd2c7cc20bca918baf7f5f861a4f10\": container with ID starting with d0fe2b8713165ec21aff402ac8185cb1d7cd2c7cc20bca918baf7f5f861a4f10 not found: ID does not exist" containerID="d0fe2b8713165ec21aff402ac8185cb1d7cd2c7cc20bca918baf7f5f861a4f10"
Sep 11 16:52:06.288707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:06.288659    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0fe2b8713165ec21aff402ac8185cb1d7cd2c7cc20bca918baf7f5f861a4f10"} err="failed to get container status \"d0fe2b8713165ec21aff402ac8185cb1d7cd2c7cc20bca918baf7f5f861a4f10\": rpc error: code = NotFound desc = could not find container \"d0fe2b8713165ec21aff402ac8185cb1d7cd2c7cc20bca918baf7f5f861a4f10\": container with ID starting with d0fe2b8713165ec21aff402ac8185cb1d7cd2c7cc20bca918baf7f5f861a4f10 not found: ID does not exist"
Sep 11 16:52:12.962630 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:12.962592    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:52:18.455013 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.454967    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr" podUID="fda59737-bb8a-48f8-baf1-6a203c120772" containerName="main" containerID="cri-o://c063ddbc4acc441e5ca9335043ac138a8080c3927a9e5dd2f268ad31b66cebf2" gracePeriod=45
Sep 11 16:52:18.716096 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.716027    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:52:18.853876 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.853834    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-tmp-dir\") pod \"fda59737-bb8a-48f8-baf1-6a203c120772\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") "
Sep 11 16:52:18.854079 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.853920    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda59737-bb8a-48f8-baf1-6a203c120772-tls-certs\") pod \"fda59737-bb8a-48f8-baf1-6a203c120772\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") "
Sep 11 16:52:18.854079 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.853958    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-model-cache\") pod \"fda59737-bb8a-48f8-baf1-6a203c120772\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") "
Sep 11 16:52:18.854197 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.854051    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-dshm\") pod \"fda59737-bb8a-48f8-baf1-6a203c120772\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") "
Sep 11 16:52:18.854197 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.854175    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7jtq\" (UniqueName: \"kubernetes.io/projected/fda59737-bb8a-48f8-baf1-6a203c120772-kube-api-access-v7jtq\") pod \"fda59737-bb8a-48f8-baf1-6a203c120772\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") "
Sep 11 16:52:18.854197 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.854173    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fda59737-bb8a-48f8-baf1-6a203c120772" (UID: "fda59737-bb8a-48f8-baf1-6a203c120772"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:52:18.854372 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.854231    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-home\") pod \"fda59737-bb8a-48f8-baf1-6a203c120772\" (UID: \"fda59737-bb8a-48f8-baf1-6a203c120772\") "
Sep 11 16:52:18.854372 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.854312    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-model-cache" (OuterVolumeSpecName: "model-cache") pod "fda59737-bb8a-48f8-baf1-6a203c120772" (UID: "fda59737-bb8a-48f8-baf1-6a203c120772"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:52:18.854519 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.854495    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-home" (OuterVolumeSpecName: "home") pod "fda59737-bb8a-48f8-baf1-6a203c120772" (UID: "fda59737-bb8a-48f8-baf1-6a203c120772"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:52:18.854637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.854611    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:52:18.854637 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.854628    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:52:18.854742 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.854643    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:52:18.856477 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.856443    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fda59737-bb8a-48f8-baf1-6a203c120772-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fda59737-bb8a-48f8-baf1-6a203c120772" (UID: "fda59737-bb8a-48f8-baf1-6a203c120772"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:52:18.856815 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.856794    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-dshm" (OuterVolumeSpecName: "dshm") pod "fda59737-bb8a-48f8-baf1-6a203c120772" (UID: "fda59737-bb8a-48f8-baf1-6a203c120772"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:52:18.856894 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.856821    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fda59737-bb8a-48f8-baf1-6a203c120772-kube-api-access-v7jtq" (OuterVolumeSpecName: "kube-api-access-v7jtq") pod "fda59737-bb8a-48f8-baf1-6a203c120772" (UID: "fda59737-bb8a-48f8-baf1-6a203c120772"). InnerVolumeSpecName "kube-api-access-v7jtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:52:18.955631 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.955584    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7jtq\" (UniqueName: \"kubernetes.io/projected/fda59737-bb8a-48f8-baf1-6a203c120772-kube-api-access-v7jtq\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:52:18.955631 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.955621    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda59737-bb8a-48f8-baf1-6a203c120772-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:52:18.955631 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:18.955632    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fda59737-bb8a-48f8-baf1-6a203c120772-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:52:19.357078 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.357040    2545 generic.go:356] "Generic (PLEG): container finished" podID="fda59737-bb8a-48f8-baf1-6a203c120772" containerID="c063ddbc4acc441e5ca9335043ac138a8080c3927a9e5dd2f268ad31b66cebf2" exitCode=0
Sep 11 16:52:19.357299 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.357084    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"
Sep 11 16:52:19.357299 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.357130    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr" event={"ID":"fda59737-bb8a-48f8-baf1-6a203c120772","Type":"ContainerDied","Data":"c063ddbc4acc441e5ca9335043ac138a8080c3927a9e5dd2f268ad31b66cebf2"}
Sep 11 16:52:19.357299 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.357182    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr" event={"ID":"fda59737-bb8a-48f8-baf1-6a203c120772","Type":"ContainerDied","Data":"cddc38c78ed0c5cd7e9aa4606721755f11435e17b7e60dfa8844c486c13aa1f2"}
Sep 11 16:52:19.357299 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.357207    2545 scope.go:117] "RemoveContainer" containerID="c063ddbc4acc441e5ca9335043ac138a8080c3927a9e5dd2f268ad31b66cebf2"
Sep 11 16:52:19.374612 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.374594    2545 scope.go:117] "RemoveContainer" containerID="c063ddbc4acc441e5ca9335043ac138a8080c3927a9e5dd2f268ad31b66cebf2"
Sep 11 16:52:19.374970 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:52:19.374947    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c063ddbc4acc441e5ca9335043ac138a8080c3927a9e5dd2f268ad31b66cebf2\": container with ID starting with c063ddbc4acc441e5ca9335043ac138a8080c3927a9e5dd2f268ad31b66cebf2 not found: ID does not exist" containerID="c063ddbc4acc441e5ca9335043ac138a8080c3927a9e5dd2f268ad31b66cebf2"
Sep 11 16:52:19.375030 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.374983    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c063ddbc4acc441e5ca9335043ac138a8080c3927a9e5dd2f268ad31b66cebf2"} err="failed to get container status \"c063ddbc4acc441e5ca9335043ac138a8080c3927a9e5dd2f268ad31b66cebf2\": rpc error: code = NotFound desc = could not find container \"c063ddbc4acc441e5ca9335043ac138a8080c3927a9e5dd2f268ad31b66cebf2\": container with ID starting with c063ddbc4acc441e5ca9335043ac138a8080c3927a9e5dd2f268ad31b66cebf2 not found: ID does not exist"
Sep 11 16:52:19.394942 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.394912    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"]
Sep 11 16:52:19.401178 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.401146    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/auth-test-model-ff7416-kserve-99b8cd4c-sfbmr"]
Sep 11 16:52:19.427855 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.427798    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f" podUID="915c120b-9044-45b1-88c1-b1c309ce5127" containerName="main" containerID="cri-o://602d88daba5d56171a457de876529514a0ebfadd936a4242c37664177e2ec8b5" gracePeriod=45
Sep 11 16:52:19.693232 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.693205    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:52:19.865756 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.865716    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-dshm\") pod \"915c120b-9044-45b1-88c1-b1c309ce5127\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") "
Sep 11 16:52:19.865958 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.865775    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/915c120b-9044-45b1-88c1-b1c309ce5127-tls-certs\") pod \"915c120b-9044-45b1-88c1-b1c309ce5127\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") "
Sep 11 16:52:19.865958 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.865826    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r6ms\" (UniqueName: \"kubernetes.io/projected/915c120b-9044-45b1-88c1-b1c309ce5127-kube-api-access-8r6ms\") pod \"915c120b-9044-45b1-88c1-b1c309ce5127\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") "
Sep 11 16:52:19.865958 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.865901    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-home\") pod \"915c120b-9044-45b1-88c1-b1c309ce5127\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") "
Sep 11 16:52:19.866149 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.866029    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-model-cache\") pod \"915c120b-9044-45b1-88c1-b1c309ce5127\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") "
Sep 11 16:52:19.866149 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.866068    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-tmp-dir\") pod \"915c120b-9044-45b1-88c1-b1c309ce5127\" (UID: \"915c120b-9044-45b1-88c1-b1c309ce5127\") "
Sep 11 16:52:19.866290 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.866257    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-home" (OuterVolumeSpecName: "home") pod "915c120b-9044-45b1-88c1-b1c309ce5127" (UID: "915c120b-9044-45b1-88c1-b1c309ce5127"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:52:19.866351 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.866283    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-model-cache" (OuterVolumeSpecName: "model-cache") pod "915c120b-9044-45b1-88c1-b1c309ce5127" (UID: "915c120b-9044-45b1-88c1-b1c309ce5127"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:52:19.866409 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.866358    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "915c120b-9044-45b1-88c1-b1c309ce5127" (UID: "915c120b-9044-45b1-88c1-b1c309ce5127"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:52:19.866519 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.866496    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:52:19.866697 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.866604    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:52:19.866750 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.866705    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:52:19.868288 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.868259    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-dshm" (OuterVolumeSpecName: "dshm") pod "915c120b-9044-45b1-88c1-b1c309ce5127" (UID: "915c120b-9044-45b1-88c1-b1c309ce5127"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:52:19.868418 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.868319    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/915c120b-9044-45b1-88c1-b1c309ce5127-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "915c120b-9044-45b1-88c1-b1c309ce5127" (UID: "915c120b-9044-45b1-88c1-b1c309ce5127"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:52:19.868418 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.868351    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/915c120b-9044-45b1-88c1-b1c309ce5127-kube-api-access-8r6ms" (OuterVolumeSpecName: "kube-api-access-8r6ms") pod "915c120b-9044-45b1-88c1-b1c309ce5127" (UID: "915c120b-9044-45b1-88c1-b1c309ce5127"). InnerVolumeSpecName "kube-api-access-8r6ms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:52:19.938370 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.938277    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fda59737-bb8a-48f8-baf1-6a203c120772" path="/var/lib/kubelet/pods/fda59737-bb8a-48f8-baf1-6a203c120772/volumes"
Sep 11 16:52:19.967758 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.967718    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/915c120b-9044-45b1-88c1-b1c309ce5127-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:52:19.967758 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.967748    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/915c120b-9044-45b1-88c1-b1c309ce5127-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:52:19.967758 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:19.967760    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r6ms\" (UniqueName: \"kubernetes.io/projected/915c120b-9044-45b1-88c1-b1c309ce5127-kube-api-access-8r6ms\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:52:20.368792 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:20.368755    2545 generic.go:356] "Generic (PLEG): container finished" podID="915c120b-9044-45b1-88c1-b1c309ce5127" containerID="602d88daba5d56171a457de876529514a0ebfadd936a4242c37664177e2ec8b5" exitCode=0
Sep 11 16:52:20.369023 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:20.368804    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"
Sep 11 16:52:20.369023 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:20.368827    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f" event={"ID":"915c120b-9044-45b1-88c1-b1c309ce5127","Type":"ContainerDied","Data":"602d88daba5d56171a457de876529514a0ebfadd936a4242c37664177e2ec8b5"}
Sep 11 16:52:20.369023 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:20.368905    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f" event={"ID":"915c120b-9044-45b1-88c1-b1c309ce5127","Type":"ContainerDied","Data":"c1384c218e9a04d51ab059675fe1cf9c22c80d9450e31aed64251d8ed3eba62e"}
Sep 11 16:52:20.369023 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:20.368927    2545 scope.go:117] "RemoveContainer" containerID="602d88daba5d56171a457de876529514a0ebfadd936a4242c37664177e2ec8b5"
Sep 11 16:52:20.379170 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:20.379146    2545 scope.go:117] "RemoveContainer" containerID="602d88daba5d56171a457de876529514a0ebfadd936a4242c37664177e2ec8b5"
Sep 11 16:52:20.379478 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:52:20.379455    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"602d88daba5d56171a457de876529514a0ebfadd936a4242c37664177e2ec8b5\": container with ID starting with 602d88daba5d56171a457de876529514a0ebfadd936a4242c37664177e2ec8b5 not found: ID does not exist" containerID="602d88daba5d56171a457de876529514a0ebfadd936a4242c37664177e2ec8b5"
Sep 11 16:52:20.379563 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:20.379491    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602d88daba5d56171a457de876529514a0ebfadd936a4242c37664177e2ec8b5"} err="failed to get container status \"602d88daba5d56171a457de876529514a0ebfadd936a4242c37664177e2ec8b5\": rpc error: code = NotFound desc = could not find container \"602d88daba5d56171a457de876529514a0ebfadd936a4242c37664177e2ec8b5\": container with ID starting with 602d88daba5d56171a457de876529514a0ebfadd936a4242c37664177e2ec8b5 not found: ID does not exist"
Sep 11 16:52:20.392390 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:20.392362    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"]
Sep 11 16:52:20.399939 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:20.399915    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/auth-test-model-ce902a-kserve-c59957d7-vpf5f"]
Sep 11 16:52:21.938848 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:21.938816    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="915c120b-9044-45b1-88c1-b1c309ce5127" path="/var/lib/kubelet/pods/915c120b-9044-45b1-88c1-b1c309ce5127/volumes"
Sep 11 16:52:25.822591 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.822555    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"]
Sep 11 16:52:25.823018 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.822973    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="915c120b-9044-45b1-88c1-b1c309ce5127" containerName="main"
Sep 11 16:52:25.823018 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.822985    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="915c120b-9044-45b1-88c1-b1c309ce5127" containerName="main"
Sep 11 16:52:25.823018 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.823009    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fda59737-bb8a-48f8-baf1-6a203c120772" containerName="main"
Sep 11 16:52:25.823018 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.823014    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="fda59737-bb8a-48f8-baf1-6a203c120772" containerName="main"
Sep 11 16:52:25.823187 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.823026    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5e18506-dbdb-45c0-8f77-a415da12a68a" containerName="test-curl-7783-f665b1"
Sep 11 16:52:25.823187 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.823032    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5e18506-dbdb-45c0-8f77-a415da12a68a" containerName="test-curl-7783-f665b1"
Sep 11 16:52:25.823187 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.823104    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="915c120b-9044-45b1-88c1-b1c309ce5127" containerName="main"
Sep 11 16:52:25.823187 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.823113    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5e18506-dbdb-45c0-8f77-a415da12a68a" containerName="test-curl-7783-f665b1"
Sep 11 16:52:25.823187 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.823123    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="fda59737-bb8a-48f8-baf1-6a203c120772" containerName="main"
Sep 11 16:52:25.827732 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.827713    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:25.831247 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.831218    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ff7416\"/\"sub-test-model-ff7416-kserve-self-signed-certs\""
Sep 11 16:52:25.831396 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.831269    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ff7416\"/\"kube-root-ca.crt\""
Sep 11 16:52:25.831396 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.831277    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ff7416\"/\"default-dockercfg-2nm4h\""
Sep 11 16:52:25.831396 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.831296    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ff7416\"/\"openshift-service-ca.crt\""
Sep 11 16:52:25.844326 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.844294    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"]
Sep 11 16:52:25.921436 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.921401    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-tmp-dir\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:25.921600 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.921449    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-home\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:25.921600 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.921481    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-model-cache\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:25.921600 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.921531    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-dshm\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:25.921600 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.921564    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-699f8\" (UniqueName: \"kubernetes.io/projected/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-kube-api-access-699f8\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:25.921743 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:25.921604    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-tls-certs\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:26.022103 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:26.022072    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-home\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:26.022323 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:26.022128    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-model-cache\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:26.022323 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:26.022160    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-dshm\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:26.022323 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:26.022209    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-699f8\" (UniqueName: \"kubernetes.io/projected/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-kube-api-access-699f8\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:26.022323 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:26.022254    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-tls-certs\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:26.022323 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:26.022310    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-tmp-dir\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:26.022587 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:26.022539    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-home\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:26.022647 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:26.022582    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-model-cache\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:26.022705 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:26.022662    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-tmp-dir\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:26.024931 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:26.024904    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-dshm\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:26.025298 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:26.025275    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-tls-certs\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:26.033738 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:26.033709    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-699f8\" (UniqueName: \"kubernetes.io/projected/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-kube-api-access-699f8\") pod \"sub-test-model-ff7416-kserve-8885c8799-zbgdn\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:26.137582 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:26.137487    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:26.426274 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:26.426239    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"]
Sep 11 16:52:26.428058 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:52:26.428031    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10b3bf6b_25dc_4978_8ef9_a07693a3a76e.slice/crio-978798ffe61f19f04bc098aa625a506174c87b9958f56a60630d7ad971847360 WatchSource:0}: Error finding container 978798ffe61f19f04bc098aa625a506174c87b9958f56a60630d7ad971847360: Status 404 returned error can't find the container with id 978798ffe61f19f04bc098aa625a506174c87b9958f56a60630d7ad971847360
Sep 11 16:52:27.414193 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:27.414152    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn" event={"ID":"10b3bf6b-25dc-4978-8ef9-a07693a3a76e","Type":"ContainerStarted","Data":"decf985b4fb77836ec174fbfe0ac552310f9d623f04d7213942f0d00cc15578b"}
Sep 11 16:52:27.414193 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:27.414196    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn" event={"ID":"10b3bf6b-25dc-4978-8ef9-a07693a3a76e","Type":"ContainerStarted","Data":"978798ffe61f19f04bc098aa625a506174c87b9958f56a60630d7ad971847360"}
Sep 11 16:52:27.432854 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:27.432804    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn" podStartSLOduration=2.432787028 podStartE2EDuration="2.432787028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:52:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:52:27.431084206 +0000 UTC m=+1365.984254144" watchObservedRunningTime="2026-09-11 16:52:27.432787028 +0000 UTC m=+1365.985956959"
Sep 11 16:52:29.876055 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:29.876009    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:52:36.138055 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:36.138014    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:36.138456 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:36.138083    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:36.150885 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:36.150830    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:36.477775 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:36.477679    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:52:46.659477 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.659440    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"]
Sep 11 16:52:46.664404 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.664383    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.666675 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.666654    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-01987b-openshift-default-dockercfg-qpjf8\""
Sep 11 16:52:46.678728 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.678706    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"]
Sep 11 16:52:46.724435 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.724406    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-workload-certs\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.724582 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.724444    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-credential-socket\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.724582 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.724490    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-workload-socket\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.724582 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.724521    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-podinfo\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.724731 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.724600    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-token\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.724731 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.724656    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-envoy\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.724731 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.724679    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-data\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.724848 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.724759    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istiod-ca-cert\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.724848 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.724789    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc7hq\" (UniqueName: \"kubernetes.io/projected/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-kube-api-access-kc7hq\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.826084 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.826044    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-envoy\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.826084 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.826085    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-data\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.826324 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.826139    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istiod-ca-cert\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.826324 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.826162    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc7hq\" (UniqueName: \"kubernetes.io/projected/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-kube-api-access-kc7hq\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.826324 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.826200    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-workload-certs\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.826324 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.826230    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-credential-socket\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.826324 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.826282    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-workload-socket\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.826324 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.826315    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-podinfo\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.826619 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.826367    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-token\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.826760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.826706    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-credential-socket\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.827080 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.826763    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-workload-socket\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.827080 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.826844    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-data\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.827080 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.826952    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-workload-certs\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.827080 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.826985    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istiod-ca-cert\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.828657 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.828634    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-envoy\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.829186 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.829167    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-podinfo\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.837257 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.837234    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc7hq\" (UniqueName: \"kubernetes.io/projected/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-kube-api-access-kc7hq\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.840264 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.840242    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-token\") pod \"e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") " pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:46.973994 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:46.973941    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:47.283530 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:47.283506    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"]
Sep 11 16:52:47.285720 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:52:47.285688    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eeafa2d_7736_4cb9_96c8_1b822270dcb4.slice/crio-74bae0f9804600e8bb6d78d31231b512b4ef780dc444489c3722c250a6ef2be3 WatchSource:0}: Error finding container 74bae0f9804600e8bb6d78d31231b512b4ef780dc444489c3722c250a6ef2be3: Status 404 returned error can't find the container with id 74bae0f9804600e8bb6d78d31231b512b4ef780dc444489c3722c250a6ef2be3
Sep 11 16:52:47.290891 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:47.288690    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:52:47.290891 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:47.288840    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:52:47.290891 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:47.288948    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:52:47.534009 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:47.533920    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" event={"ID":"4eeafa2d-7736-4cb9-96c8-1b822270dcb4","Type":"ContainerStarted","Data":"00ca6b5588972d593851a2ec0e76091817fa3da439f472617fdffcafad524eb9"}
Sep 11 16:52:47.534166 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:47.534007    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" event={"ID":"4eeafa2d-7736-4cb9-96c8-1b822270dcb4","Type":"ContainerStarted","Data":"74bae0f9804600e8bb6d78d31231b512b4ef780dc444489c3722c250a6ef2be3"}
Sep 11 16:52:47.561820 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:47.561762    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" podStartSLOduration=1.561742228 podStartE2EDuration="1.561742228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:52:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:52:47.558614498 +0000 UTC m=+1386.111784437" watchObservedRunningTime="2026-09-11 16:52:47.561742228 +0000 UTC m=+1386.114912158"
Sep 11 16:52:47.974269 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:47.974185    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:48.979819 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:48.979787    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:49.546188 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:49.546152    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:49.547091 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:49.547073    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:52:56.053521 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:52:56.053480    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:53:05.588035 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:05.587991    2545 patch_prober.go:28] interesting pod/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 16:53:05.588419 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:05.588087    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 16:53:06.653330 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:06.653297    2545 generic.go:356] "Generic (PLEG): container finished" podID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerID="00ca6b5588972d593851a2ec0e76091817fa3da439f472617fdffcafad524eb9" exitCode=0
Sep 11 16:53:06.653787 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:06.653373    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" event={"ID":"4eeafa2d-7736-4cb9-96c8-1b822270dcb4","Type":"ContainerDied","Data":"00ca6b5588972d593851a2ec0e76091817fa3da439f472617fdffcafad524eb9"}
Sep 11 16:53:06.653837 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:06.653790    2545 scope.go:117] "RemoveContainer" containerID="00ca6b5588972d593851a2ec0e76091817fa3da439f472617fdffcafad524eb9"
Sep 11 16:53:06.654972 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:06.654939    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:53:06.655086 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:06.655032    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:53:06.655152 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:06.655085    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:53:07.660613 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:07.660577    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" event={"ID":"4eeafa2d-7736-4cb9-96c8-1b822270dcb4","Type":"ContainerStarted","Data":"b0b2c7d4d043de4a297473a8aa98578ad93e2d6c75ca0789b3d40a46649ff6a3"}
Sep 11 16:53:07.974640 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:07.974548    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:53:07.974790 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:07.974723    2545 patch_prober.go:28] interesting pod/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 11 16:53:07.974843 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:07.974792    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 11 16:53:08.974814 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:08.974780    2545 patch_prober.go:28] interesting pod/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 11 16:53:08.975327 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:08.974837    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 11 16:53:09.975230 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:09.975195    2545 patch_prober.go:28] interesting pod/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 11 16:53:09.975819 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:09.975783    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 11 16:53:10.974676 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:10.974641    2545 patch_prober.go:28] interesting pod/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 11 16:53:10.974857 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:10.974692    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 11 16:53:11.975286 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:11.975248    2545 patch_prober.go:28] interesting pod/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 11 16:53:11.975929 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:11.975894    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 11 16:53:12.974837 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:12.974799    2545 patch_prober.go:28] interesting pod/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 11 16:53:12.975097 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:12.974853    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 11 16:53:13.998416 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:13.998384    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:53:13.998798 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:13.998649    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:53:13.999464 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:13.999446    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:53:15.432216 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.432183    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"]
Sep 11 16:53:15.435978 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.435956    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.438338 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.438313    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-16265e-openshift-default-dockercfg-nt8s5\""
Sep 11 16:53:15.451711 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.451683    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"]
Sep 11 16:53:15.527414 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.527356    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-workload-socket\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.527631 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.527474    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-workload-certs\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.527631 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.527543    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istiod-ca-cert\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.527771 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.527643    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84qwg\" (UniqueName: \"kubernetes.io/projected/b831ab69-7663-4c4d-bc43-d47f0393a8b6-kube-api-access-84qwg\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.527771 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.527701    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-token\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.527919 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.527781    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-envoy\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.527919 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.527819    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-podinfo\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.528048 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.528026    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-data\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.528109 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.528075    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-credential-socket\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.629569 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.629537    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istiod-ca-cert\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.629771 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.629617    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84qwg\" (UniqueName: \"kubernetes.io/projected/b831ab69-7663-4c4d-bc43-d47f0393a8b6-kube-api-access-84qwg\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.629771 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.629648    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-token\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.629922 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.629776    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-envoy\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.629922 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.629831    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-podinfo\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.629922 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.629904    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-data\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.630087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.629938    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-credential-socket\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.630087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.629994    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-workload-socket\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.630087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.630031    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-workload-certs\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.630369 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.630347    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istiod-ca-cert\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.630716 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.630361    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-data\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.630716 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.630385    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-credential-socket\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.630846 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.630415    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-workload-socket\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.630846 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.630583    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-workload-certs\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.632320 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.632299    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-envoy\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.632515 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.632493    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-podinfo\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.639890 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.639836    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-token\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.642771 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.642744    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84qwg\" (UniqueName: \"kubernetes.io/projected/b831ab69-7663-4c4d-bc43-d47f0393a8b6-kube-api-access-84qwg\") pod \"e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") " pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:15.748278 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:15.748256    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:16.034571 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:16.034543    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"]
Sep 11 16:53:16.036297 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:53:16.036266    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb831ab69_7663_4c4d_bc43_d47f0393a8b6.slice/crio-19ae16c83739524458d2a707f2cf7a920869f8129a0a031ff99bc904601ee5e4 WatchSource:0}: Error finding container 19ae16c83739524458d2a707f2cf7a920869f8129a0a031ff99bc904601ee5e4: Status 404 returned error can't find the container with id 19ae16c83739524458d2a707f2cf7a920869f8129a0a031ff99bc904601ee5e4
Sep 11 16:53:16.038523 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:16.038490    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:53:16.038634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:16.038568    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:53:16.038634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:16.038611    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:53:16.728208 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:16.728171    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj" event={"ID":"b831ab69-7663-4c4d-bc43-d47f0393a8b6","Type":"ContainerStarted","Data":"24feb6e6ba2150226c5f88c90306ac5beaf06e3a5ad7e4d4c21edb263679042e"}
Sep 11 16:53:16.728600 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:16.728237    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj" event={"ID":"b831ab69-7663-4c4d-bc43-d47f0393a8b6","Type":"ContainerStarted","Data":"19ae16c83739524458d2a707f2cf7a920869f8129a0a031ff99bc904601ee5e4"}
Sep 11 16:53:16.748465 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:16.748441    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:16.763071 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:16.763030    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj" podStartSLOduration=1.763015549 podStartE2EDuration="1.763015549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:53:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:53:16.761094648 +0000 UTC m=+1415.314264606" watchObservedRunningTime="2026-09-11 16:53:16.763015549 +0000 UTC m=+1415.316185476"
Sep 11 16:53:17.754668 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:17.754639    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:18.739010 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:18.738984    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:18.739784 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:18.739765    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:28.012790 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.012756    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"]
Sep 11 16:53:28.016305 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.016282    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.018440 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.018420    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-ce902a\"/\"default-dockercfg-vj4dp\""
Sep 11 16:53:28.019086 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.019067    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-ce902a\"/\"openshift-service-ca.crt\""
Sep 11 16:53:28.019195 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.019091    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-ce902a\"/\"sub-test-model-ce902a-kserve-self-signed-certs\""
Sep 11 16:53:28.019303 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.019284    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-ce902a\"/\"kube-root-ca.crt\""
Sep 11 16:53:28.028709 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.028690    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"]
Sep 11 16:53:28.153411 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.153384    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-tmp-dir\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.153558 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.153454    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-tls-certs\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.153558 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.153519    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-home\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.153650 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.153577    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-model-cache\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.153650 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.153595    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thv6z\" (UniqueName: \"kubernetes.io/projected/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-kube-api-access-thv6z\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.153731 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.153649    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-dshm\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.254938 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.254902    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-tls-certs\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.255157 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.254959    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-home\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.255157 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.255018    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-model-cache\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.255157 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.255039    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thv6z\" (UniqueName: \"kubernetes.io/projected/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-kube-api-access-thv6z\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.255339 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.255187    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-dshm\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.255339 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.255268    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-tmp-dir\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.255474 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.255455    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-model-cache\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.255528 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.255507    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-home\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.255646 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.255627    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-tmp-dir\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.257533 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.257511    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-dshm\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.257763 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.257745    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-tls-certs\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.269928 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.269856    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thv6z\" (UniqueName: \"kubernetes.io/projected/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-kube-api-access-thv6z\") pod \"sub-test-model-ce902a-kserve-7b58bf7767-78l7k\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.327681 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.327657    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:28.609591 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.609556    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"]
Sep 11 16:53:28.611359 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:53:28.611334    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2d37ed4_fd9f_48d0_9825_b1165a4b42f6.slice/crio-b3e8a37c90579642a7c2446ccf2eac735b96644b19c1fd0aaf1b73d5e411804b WatchSource:0}: Error finding container b3e8a37c90579642a7c2446ccf2eac735b96644b19c1fd0aaf1b73d5e411804b: Status 404 returned error can't find the container with id b3e8a37c90579642a7c2446ccf2eac735b96644b19c1fd0aaf1b73d5e411804b
Sep 11 16:53:28.798129 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.798090    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k" event={"ID":"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6","Type":"ContainerStarted","Data":"0f5353b978d9c5d933927eeca2288ff3689038a7254aab4ce1e7e1d435d8f879"}
Sep 11 16:53:28.798129 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.798134    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k" event={"ID":"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6","Type":"ContainerStarted","Data":"b3e8a37c90579642a7c2446ccf2eac735b96644b19c1fd0aaf1b73d5e411804b"}
Sep 11 16:53:28.824224 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:28.824175    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k" podStartSLOduration=1.8241603020000001 podStartE2EDuration="1.824160302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:53:28.823904999 +0000 UTC m=+1427.377074943" watchObservedRunningTime="2026-09-11 16:53:28.824160302 +0000 UTC m=+1427.377330228"
Sep 11 16:53:31.664359 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:31.664327    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:53:38.328086 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:38.327985    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:38.328086 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:38.328045    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:38.347001 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:38.346972    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:38.866026 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:38.865999    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:53:41.876813 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:41.876720    2545 generic.go:356] "Generic (PLEG): container finished" podID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" containerID="24feb6e6ba2150226c5f88c90306ac5beaf06e3a5ad7e4d4c21edb263679042e" exitCode=0
Sep 11 16:53:41.876813 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:41.876753    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj" event={"ID":"b831ab69-7663-4c4d-bc43-d47f0393a8b6","Type":"ContainerDied","Data":"24feb6e6ba2150226c5f88c90306ac5beaf06e3a5ad7e4d4c21edb263679042e"}
Sep 11 16:53:41.877416 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:41.877103    2545 scope.go:117] "RemoveContainer" containerID="24feb6e6ba2150226c5f88c90306ac5beaf06e3a5ad7e4d4c21edb263679042e"
Sep 11 16:53:41.878408 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:41.878378    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:53:41.878492 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:41.878448    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:53:41.878552 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:41.878498    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:53:42.311879 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:42.311837    2545 scope.go:117] "RemoveContainer" containerID="57068e166f319d759a2d1fcde7bfeb2595fdd5c634a21eb65db392f60fae39d8"
Sep 11 16:53:42.320937 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:42.320914    2545 scope.go:117] "RemoveContainer" containerID="ab79f6940365a0b251063d266f1f1f86f13aa58af72bb12cbcd76d716e0f71f0"
Sep 11 16:53:42.888475 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:42.888426    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj" event={"ID":"b831ab69-7663-4c4d-bc43-d47f0393a8b6","Type":"ContainerStarted","Data":"d50e1c20046ca417a2fea7c71066e9cb40663da89500a19b6c1ee1cde7e5e9be"}
Sep 11 16:53:43.749052 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:43.749015    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:43.749275 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:43.749252    2545 patch_prober.go:28] interesting pod/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Sep 11 16:53:43.749341 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:43.749321    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj" podUID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Sep 11 16:53:44.749174 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:44.749134    2545 patch_prober.go:28] interesting pod/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Sep 11 16:53:44.749616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:44.749222    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj" podUID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Sep 11 16:53:45.749130 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:45.749093    2545 patch_prober.go:28] interesting pod/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Sep 11 16:53:45.749326 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:45.749152    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj" podUID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Sep 11 16:53:46.727575 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:46.727542    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk"]
Sep 11 16:53:46.731222 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:46.731199    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk"
Sep 11 16:53:46.733616 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:46.733587    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:53:46.733756 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:46.733696    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:53:46.740165 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:46.740140    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk"]
Sep 11 16:53:46.749291 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:46.749256    2545 patch_prober.go:28] interesting pod/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Sep 11 16:53:46.749423 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:46.749321    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj" podUID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Sep 11 16:53:46.754370 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:46.754337    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39180e6a-62f6-496f-b054-b2ba13b6a535-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-01987b-fh4xk\" (UID: \"39180e6a-62f6-496f-b054-b2ba13b6a535\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk"
Sep 11 16:53:46.855943 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:46.855897    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39180e6a-62f6-496f-b054-b2ba13b6a535-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-01987b-fh4xk\" (UID: \"39180e6a-62f6-496f-b054-b2ba13b6a535\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk"
Sep 11 16:53:46.856490 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:46.856469    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39180e6a-62f6-496f-b054-b2ba13b6a535-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-01987b-fh4xk\" (UID: \"39180e6a-62f6-496f-b054-b2ba13b6a535\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk"
Sep 11 16:53:47.044208 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:47.044167    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk"
Sep 11 16:53:47.349629 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:47.349599    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk"]
Sep 11 16:53:47.351544 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:53:47.351517    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39180e6a_62f6_496f_b054_b2ba13b6a535.slice/crio-34bfe0882dd0f3fca90d317674b7bc79e47c3a62a2c7f62e99c871375c5c1e7d WatchSource:0}: Error finding container 34bfe0882dd0f3fca90d317674b7bc79e47c3a62a2c7f62e99c871375c5c1e7d: Status 404 returned error can't find the container with id 34bfe0882dd0f3fca90d317674b7bc79e47c3a62a2c7f62e99c871375c5c1e7d
Sep 11 16:53:47.749304 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:47.749262    2545 patch_prober.go:28] interesting pod/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Sep 11 16:53:47.749494 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:47.749334    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj" podUID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Sep 11 16:53:47.925825 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:47.925715    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk" event={"ID":"39180e6a-62f6-496f-b054-b2ba13b6a535","Type":"ContainerStarted","Data":"36c0c82b5fcddaf499609b3c631e791366dd693f54837db9aaa36888a009334f"}
Sep 11 16:53:47.925825 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:47.925775    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk" event={"ID":"39180e6a-62f6-496f-b054-b2ba13b6a535","Type":"ContainerStarted","Data":"34bfe0882dd0f3fca90d317674b7bc79e47c3a62a2c7f62e99c871375c5c1e7d"}
Sep 11 16:53:47.948359 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:47.948298    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk" podStartSLOduration=1.948282137 podStartE2EDuration="1.948282137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:53:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:53:47.946390992 +0000 UTC m=+1446.499560920" watchObservedRunningTime="2026-09-11 16:53:47.948282137 +0000 UTC m=+1446.501452063"
Sep 11 16:53:48.739779 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:48.739745    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:48.753778 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:48.753745    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:48.932949 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:48.932913    2545 generic.go:356] "Generic (PLEG): container finished" podID="39180e6a-62f6-496f-b054-b2ba13b6a535" containerID="36c0c82b5fcddaf499609b3c631e791366dd693f54837db9aaa36888a009334f" exitCode=0
Sep 11 16:53:48.933145 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:48.933002    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk" event={"ID":"39180e6a-62f6-496f-b054-b2ba13b6a535","Type":"ContainerDied","Data":"36c0c82b5fcddaf499609b3c631e791366dd693f54837db9aaa36888a009334f"}
Sep 11 16:53:48.934088 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:48.934062    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:53:50.084526 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:50.084497    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk"
Sep 11 16:53:50.192389 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:50.192296    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39180e6a-62f6-496f-b054-b2ba13b6a535-maas-api-service-ca\") pod \"39180e6a-62f6-496f-b054-b2ba13b6a535\" (UID: \"39180e6a-62f6-496f-b054-b2ba13b6a535\") "
Sep 11 16:53:50.192688 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:50.192665    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39180e6a-62f6-496f-b054-b2ba13b6a535-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "39180e6a-62f6-496f-b054-b2ba13b6a535" (UID: "39180e6a-62f6-496f-b054-b2ba13b6a535"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:53:50.293947 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:50.293907    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39180e6a-62f6-496f-b054-b2ba13b6a535-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:53:50.946573 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:50.946531    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk" event={"ID":"39180e6a-62f6-496f-b054-b2ba13b6a535","Type":"ContainerDied","Data":"34bfe0882dd0f3fca90d317674b7bc79e47c3a62a2c7f62e99c871375c5c1e7d"}
Sep 11 16:53:50.946573 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:50.946562    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk"
Sep 11 16:53:50.946825 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:53:50.946595    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34bfe0882dd0f3fca90d317674b7bc79e47c3a62a2c7f62e99c871375c5c1e7d"
Sep 11 16:54:00.473419 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:00.473378    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:54:02.914773 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:02.914738    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk"]
Sep 11 16:54:02.917231 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:02.917209    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-01987b-fh4xk"]
Sep 11 16:54:03.938940 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:03.938905    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39180e6a-62f6-496f-b054-b2ba13b6a535" path="/var/lib/kubelet/pods/39180e6a-62f6-496f-b054-b2ba13b6a535/volumes"
Sep 11 16:54:11.034716 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:11.034684    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"]
Sep 11 16:54:11.035191 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:11.034923    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy" containerID="cri-o://b0b2c7d4d043de4a297473a8aa98578ad93e2d6c75ca0789b3d40a46649ff6a3" gracePeriod=30
Sep 11 16:54:11.415346 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:11.415251    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78"]
Sep 11 16:54:11.415702 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:11.415690    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39180e6a-62f6-496f-b054-b2ba13b6a535" containerName="revoke-keys"
Sep 11 16:54:11.415751 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:11.415705    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="39180e6a-62f6-496f-b054-b2ba13b6a535" containerName="revoke-keys"
Sep 11 16:54:11.415795 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:11.415786    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="39180e6a-62f6-496f-b054-b2ba13b6a535" containerName="revoke-keys"
Sep 11 16:54:11.419137 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:11.419115    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78"
Sep 11 16:54:11.423650 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:11.423626    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:54:11.424104 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:11.424088    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:54:11.427058 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:11.427032    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51f8d89d-3a4c-486f-b360-ce43a9add927-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-16265e-zkh78\" (UID: \"51f8d89d-3a4c-486f-b360-ce43a9add927\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78"
Sep 11 16:54:11.471811 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:11.471777    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78"]
Sep 11 16:54:11.528325 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:11.528290    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51f8d89d-3a4c-486f-b360-ce43a9add927-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-16265e-zkh78\" (UID: \"51f8d89d-3a4c-486f-b360-ce43a9add927\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78"
Sep 11 16:54:11.529218 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:11.529194    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51f8d89d-3a4c-486f-b360-ce43a9add927-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-16265e-zkh78\" (UID: \"51f8d89d-3a4c-486f-b360-ce43a9add927\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78"
Sep 11 16:54:11.729477 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:11.729381    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78"
Sep 11 16:54:12.021264 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:12.021235    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78"]
Sep 11 16:54:12.024112 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:54:12.024082    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51f8d89d_3a4c_486f_b360_ce43a9add927.slice/crio-9520a60b9fc0e5017313f3d80183f0209c6a37083a1386b3b9b212bafe066716 WatchSource:0}: Error finding container 9520a60b9fc0e5017313f3d80183f0209c6a37083a1386b3b9b212bafe066716: Status 404 returned error can't find the container with id 9520a60b9fc0e5017313f3d80183f0209c6a37083a1386b3b9b212bafe066716
Sep 11 16:54:12.074144 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:12.074113    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78" event={"ID":"51f8d89d-3a4c-486f-b360-ce43a9add927","Type":"ContainerStarted","Data":"9520a60b9fc0e5017313f3d80183f0209c6a37083a1386b3b9b212bafe066716"}
Sep 11 16:54:13.082513 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:13.082479    2545 generic.go:356] "Generic (PLEG): container finished" podID="51f8d89d-3a4c-486f-b360-ce43a9add927" containerID="787397c29501a57d1f1eaf16a877ef89cd34f771378ecba4f6a4d22f8f404dac" exitCode=0
Sep 11 16:54:13.083062 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:13.082592    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78" event={"ID":"51f8d89d-3a4c-486f-b360-ce43a9add927","Type":"ContainerDied","Data":"787397c29501a57d1f1eaf16a877ef89cd34f771378ecba4f6a4d22f8f404dac"}
Sep 11 16:54:14.001881 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:14.001815    2545 patch_prober.go:28] interesting pod/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 16:54:14.002086 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:14.001914    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 16:54:14.234351 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:14.234321    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78"
Sep 11 16:54:14.256777 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:14.256681    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51f8d89d-3a4c-486f-b360-ce43a9add927-maas-api-service-ca\") pod \"51f8d89d-3a4c-486f-b360-ce43a9add927\" (UID: \"51f8d89d-3a4c-486f-b360-ce43a9add927\") "
Sep 11 16:54:14.257087 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:14.257065    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51f8d89d-3a4c-486f-b360-ce43a9add927-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "51f8d89d-3a4c-486f-b360-ce43a9add927" (UID: "51f8d89d-3a4c-486f-b360-ce43a9add927"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:54:14.358437 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:14.358393    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51f8d89d-3a4c-486f-b360-ce43a9add927-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:15.097812 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:15.097782    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78"
Sep 11 16:54:15.098019 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:15.097818    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78" event={"ID":"51f8d89d-3a4c-486f-b360-ce43a9add927","Type":"ContainerDied","Data":"9520a60b9fc0e5017313f3d80183f0209c6a37083a1386b3b9b212bafe066716"}
Sep 11 16:54:15.098019 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:15.097891    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9520a60b9fc0e5017313f3d80183f0209c6a37083a1386b3b9b212bafe066716"
Sep 11 16:54:15.668848 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:15.668811    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7783-643e88"]
Sep 11 16:54:15.669298 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:15.669284    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51f8d89d-3a4c-486f-b360-ce43a9add927" containerName="revoke-keys"
Sep 11 16:54:15.669354 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:15.669299    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="51f8d89d-3a4c-486f-b360-ce43a9add927" containerName="revoke-keys"
Sep 11 16:54:15.669391 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:15.669378    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="51f8d89d-3a4c-486f-b360-ce43a9add927" containerName="revoke-keys"
Sep 11 16:54:15.672313 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:15.672287    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7783-643e88"
Sep 11 16:54:15.675448 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:15.675426    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vtvd2\""
Sep 11 16:54:15.689268 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:15.689240    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7783-643e88"]
Sep 11 16:54:15.773692 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:15.773649    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhzj2\" (UniqueName: \"kubernetes.io/projected/6594b0e2-09f2-4639-9f66-15ac7cc55b85-kube-api-access-qhzj2\") pod \"test-curl-7783-643e88\" (UID: \"6594b0e2-09f2-4639-9f66-15ac7cc55b85\") " pod="opendatahub/test-curl-7783-643e88"
Sep 11 16:54:15.875319 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:15.875280    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhzj2\" (UniqueName: \"kubernetes.io/projected/6594b0e2-09f2-4639-9f66-15ac7cc55b85-kube-api-access-qhzj2\") pod \"test-curl-7783-643e88\" (UID: \"6594b0e2-09f2-4639-9f66-15ac7cc55b85\") " pod="opendatahub/test-curl-7783-643e88"
Sep 11 16:54:15.885755 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:15.885716    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhzj2\" (UniqueName: \"kubernetes.io/projected/6594b0e2-09f2-4639-9f66-15ac7cc55b85-kube-api-access-qhzj2\") pod \"test-curl-7783-643e88\" (UID: \"6594b0e2-09f2-4639-9f66-15ac7cc55b85\") " pod="opendatahub/test-curl-7783-643e88"
Sep 11 16:54:15.982442 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:15.982353    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7783-643e88"
Sep 11 16:54:16.259419 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.259393    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7783-643e88"]
Sep 11 16:54:16.261382 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:54:16.261348    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6594b0e2_09f2_4639_9f66_15ac7cc55b85.slice/crio-369de27b70dbe8506b60b3b645f7f82a411b0c5ee72f6cecab10fb831c5cded0 WatchSource:0}: Error finding container 369de27b70dbe8506b60b3b645f7f82a411b0c5ee72f6cecab10fb831c5cded0: Status 404 returned error can't find the container with id 369de27b70dbe8506b60b3b645f7f82a411b0c5ee72f6cecab10fb831c5cded0
Sep 11 16:54:16.311388 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.311357    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:54:16.381009 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.380977    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-workload-socket\") pod \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") "
Sep 11 16:54:16.381216 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381052    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-envoy\") pod \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") "
Sep 11 16:54:16.381216 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381077    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-workload-certs\") pod \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") "
Sep 11 16:54:16.381216 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381125    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-token\") pod \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") "
Sep 11 16:54:16.381216 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381151    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-podinfo\") pod \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") "
Sep 11 16:54:16.381216 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381192    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-data\") pod \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") "
Sep 11 16:54:16.381494 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381237    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-credential-socket\") pod \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") "
Sep 11 16:54:16.381494 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381275    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kc7hq\" (UniqueName: \"kubernetes.io/projected/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-kube-api-access-kc7hq\") pod \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") "
Sep 11 16:54:16.381494 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381309    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istiod-ca-cert\") pod \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\" (UID: \"4eeafa2d-7736-4cb9-96c8-1b822270dcb4\") "
Sep 11 16:54:16.381494 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381307    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4eeafa2d-7736-4cb9-96c8-1b822270dcb4" (UID: "4eeafa2d-7736-4cb9-96c8-1b822270dcb4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:16.381494 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381365    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4eeafa2d-7736-4cb9-96c8-1b822270dcb4" (UID: "4eeafa2d-7736-4cb9-96c8-1b822270dcb4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:16.381757 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381565    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4eeafa2d-7736-4cb9-96c8-1b822270dcb4" (UID: "4eeafa2d-7736-4cb9-96c8-1b822270dcb4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:16.381757 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381591    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-data" (OuterVolumeSpecName: "istio-data") pod "4eeafa2d-7736-4cb9-96c8-1b822270dcb4" (UID: "4eeafa2d-7736-4cb9-96c8-1b822270dcb4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:16.381917 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381819    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:16.381917 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381848    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:16.381917 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381880    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:16.381917 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.381895    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:16.382143 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.382033    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4eeafa2d-7736-4cb9-96c8-1b822270dcb4" (UID: "4eeafa2d-7736-4cb9-96c8-1b822270dcb4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:54:16.383648 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.383624    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-token" (OuterVolumeSpecName: "istio-token") pod "4eeafa2d-7736-4cb9-96c8-1b822270dcb4" (UID: "4eeafa2d-7736-4cb9-96c8-1b822270dcb4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:54:16.383759 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.383679    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-kube-api-access-kc7hq" (OuterVolumeSpecName: "kube-api-access-kc7hq") pod "4eeafa2d-7736-4cb9-96c8-1b822270dcb4" (UID: "4eeafa2d-7736-4cb9-96c8-1b822270dcb4"). InnerVolumeSpecName "kube-api-access-kc7hq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:54:16.383806 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.383754    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4eeafa2d-7736-4cb9-96c8-1b822270dcb4" (UID: "4eeafa2d-7736-4cb9-96c8-1b822270dcb4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:54:16.384097 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.384079    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4eeafa2d-7736-4cb9-96c8-1b822270dcb4" (UID: "4eeafa2d-7736-4cb9-96c8-1b822270dcb4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:16.487219 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.487160    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kc7hq\" (UniqueName: \"kubernetes.io/projected/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-kube-api-access-kc7hq\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:16.487219 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.487203    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:16.487219 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.487224    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:16.487647 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.487235    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:16.487647 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:16.487247    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eeafa2d-7736-4cb9-96c8-1b822270dcb4-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:17.112390 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.112349    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7783-643e88" event={"ID":"6594b0e2-09f2-4639-9f66-15ac7cc55b85","Type":"ContainerStarted","Data":"54a5d4f6e407b7fb337df89bcb7ddfdadf4d4a2c4e3ebd202aa3dba8f1400888"}
Sep 11 16:54:17.112390 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.112397    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7783-643e88" event={"ID":"6594b0e2-09f2-4639-9f66-15ac7cc55b85","Type":"ContainerStarted","Data":"369de27b70dbe8506b60b3b645f7f82a411b0c5ee72f6cecab10fb831c5cded0"}
Sep 11 16:54:17.114201 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.114170    2545 generic.go:356] "Generic (PLEG): container finished" podID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerID="b0b2c7d4d043de4a297473a8aa98578ad93e2d6c75ca0789b3d40a46649ff6a3" exitCode=0
Sep 11 16:54:17.114325 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.114237    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"
Sep 11 16:54:17.114325 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.114252    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" event={"ID":"4eeafa2d-7736-4cb9-96c8-1b822270dcb4","Type":"ContainerDied","Data":"b0b2c7d4d043de4a297473a8aa98578ad93e2d6c75ca0789b3d40a46649ff6a3"}
Sep 11 16:54:17.114325 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.114306    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq" event={"ID":"4eeafa2d-7736-4cb9-96c8-1b822270dcb4","Type":"ContainerDied","Data":"74bae0f9804600e8bb6d78d31231b512b4ef780dc444489c3722c250a6ef2be3"}
Sep 11 16:54:17.114425 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.114328    2545 scope.go:117] "RemoveContainer" containerID="b0b2c7d4d043de4a297473a8aa98578ad93e2d6c75ca0789b3d40a46649ff6a3"
Sep 11 16:54:17.126730 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.126707    2545 scope.go:117] "RemoveContainer" containerID="00ca6b5588972d593851a2ec0e76091817fa3da439f472617fdffcafad524eb9"
Sep 11 16:54:17.135918 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.135893    2545 scope.go:117] "RemoveContainer" containerID="b0b2c7d4d043de4a297473a8aa98578ad93e2d6c75ca0789b3d40a46649ff6a3"
Sep 11 16:54:17.136257 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:54:17.136224    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0b2c7d4d043de4a297473a8aa98578ad93e2d6c75ca0789b3d40a46649ff6a3\": container with ID starting with b0b2c7d4d043de4a297473a8aa98578ad93e2d6c75ca0789b3d40a46649ff6a3 not found: ID does not exist" containerID="b0b2c7d4d043de4a297473a8aa98578ad93e2d6c75ca0789b3d40a46649ff6a3"
Sep 11 16:54:17.136352 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.136256    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0b2c7d4d043de4a297473a8aa98578ad93e2d6c75ca0789b3d40a46649ff6a3"} err="failed to get container status \"b0b2c7d4d043de4a297473a8aa98578ad93e2d6c75ca0789b3d40a46649ff6a3\": rpc error: code = NotFound desc = could not find container \"b0b2c7d4d043de4a297473a8aa98578ad93e2d6c75ca0789b3d40a46649ff6a3\": container with ID starting with b0b2c7d4d043de4a297473a8aa98578ad93e2d6c75ca0789b3d40a46649ff6a3 not found: ID does not exist"
Sep 11 16:54:17.136352 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.136275    2545 scope.go:117] "RemoveContainer" containerID="00ca6b5588972d593851a2ec0e76091817fa3da439f472617fdffcafad524eb9"
Sep 11 16:54:17.136593 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:54:17.136576    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00ca6b5588972d593851a2ec0e76091817fa3da439f472617fdffcafad524eb9\": container with ID starting with 00ca6b5588972d593851a2ec0e76091817fa3da439f472617fdffcafad524eb9 not found: ID does not exist" containerID="00ca6b5588972d593851a2ec0e76091817fa3da439f472617fdffcafad524eb9"
Sep 11 16:54:17.136636 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.136600    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00ca6b5588972d593851a2ec0e76091817fa3da439f472617fdffcafad524eb9"} err="failed to get container status \"00ca6b5588972d593851a2ec0e76091817fa3da439f472617fdffcafad524eb9\": rpc error: code = NotFound desc = could not find container \"00ca6b5588972d593851a2ec0e76091817fa3da439f472617fdffcafad524eb9\": container with ID starting with 00ca6b5588972d593851a2ec0e76091817fa3da439f472617fdffcafad524eb9 not found: ID does not exist"
Sep 11 16:54:17.141740 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.141693    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7783-643e88" podStartSLOduration=2.141679424 podStartE2EDuration="2.141679424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:54:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:54:17.140239929 +0000 UTC m=+1475.693409856" watchObservedRunningTime="2026-09-11 16:54:17.141679424 +0000 UTC m=+1475.694849386"
Sep 11 16:54:17.170714 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.170623    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"]
Sep 11 16:54:17.177285 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.177253    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-01987b-openshift-default-65b86d8dc4-x9wnq"]
Sep 11 16:54:17.801199 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.801158    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7783-643e88"]
Sep 11 16:54:17.808270 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.808240    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7783-643e88"]
Sep 11 16:54:17.938388 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:17.938301    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" path="/var/lib/kubelet/pods/4eeafa2d-7736-4cb9-96c8-1b822270dcb4/volumes"
Sep 11 16:54:18.121469 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:54:18.121443    2545 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7783-643e88" container="test-curl-7783-643e88"
Sep 11 16:54:19.127277 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:19.127237    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7783-643e88" podUID="6594b0e2-09f2-4639-9f66-15ac7cc55b85" containerName="test-curl-7783-643e88" containerID="cri-o://54a5d4f6e407b7fb337df89bcb7ddfdadf4d4a2c4e3ebd202aa3dba8f1400888" gracePeriod=2
Sep 11 16:54:19.447941 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:19.447832    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:54:19.909225 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:19.909193    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"]
Sep 11 16:54:20.866651 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:20.866619    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"]
Sep 11 16:54:21.302345 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:21.302325    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7783-643e88"
Sep 11 16:54:21.439370 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:21.439280    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhzj2\" (UniqueName: \"kubernetes.io/projected/6594b0e2-09f2-4639-9f66-15ac7cc55b85-kube-api-access-qhzj2\") pod \"6594b0e2-09f2-4639-9f66-15ac7cc55b85\" (UID: \"6594b0e2-09f2-4639-9f66-15ac7cc55b85\") "
Sep 11 16:54:21.441514 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:21.441491    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6594b0e2-09f2-4639-9f66-15ac7cc55b85-kube-api-access-qhzj2" (OuterVolumeSpecName: "kube-api-access-qhzj2") pod "6594b0e2-09f2-4639-9f66-15ac7cc55b85" (UID: "6594b0e2-09f2-4639-9f66-15ac7cc55b85"). InnerVolumeSpecName "kube-api-access-qhzj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:54:21.540829 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:21.540791    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhzj2\" (UniqueName: \"kubernetes.io/projected/6594b0e2-09f2-4639-9f66-15ac7cc55b85-kube-api-access-qhzj2\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:21.937754 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:21.937723    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6594b0e2-09f2-4639-9f66-15ac7cc55b85" path="/var/lib/kubelet/pods/6594b0e2-09f2-4639-9f66-15ac7cc55b85/volumes"
Sep 11 16:54:22.145855 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:22.145807    2545 generic.go:356] "Generic (PLEG): container finished" podID="6594b0e2-09f2-4639-9f66-15ac7cc55b85" containerID="54a5d4f6e407b7fb337df89bcb7ddfdadf4d4a2c4e3ebd202aa3dba8f1400888" exitCode=137
Sep 11 16:54:22.146030 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:22.145914    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7783-643e88"
Sep 11 16:54:22.146030 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:22.145915    2545 scope.go:117] "RemoveContainer" containerID="54a5d4f6e407b7fb337df89bcb7ddfdadf4d4a2c4e3ebd202aa3dba8f1400888"
Sep 11 16:54:22.154813 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:22.154795    2545 scope.go:117] "RemoveContainer" containerID="54a5d4f6e407b7fb337df89bcb7ddfdadf4d4a2c4e3ebd202aa3dba8f1400888"
Sep 11 16:54:22.155117 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:54:22.155096    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54a5d4f6e407b7fb337df89bcb7ddfdadf4d4a2c4e3ebd202aa3dba8f1400888\": container with ID starting with 54a5d4f6e407b7fb337df89bcb7ddfdadf4d4a2c4e3ebd202aa3dba8f1400888 not found: ID does not exist" containerID="54a5d4f6e407b7fb337df89bcb7ddfdadf4d4a2c4e3ebd202aa3dba8f1400888"
Sep 11 16:54:22.155187 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:22.155128    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a5d4f6e407b7fb337df89bcb7ddfdadf4d4a2c4e3ebd202aa3dba8f1400888"} err="failed to get container status \"54a5d4f6e407b7fb337df89bcb7ddfdadf4d4a2c4e3ebd202aa3dba8f1400888\": rpc error: code = NotFound desc = could not find container \"54a5d4f6e407b7fb337df89bcb7ddfdadf4d4a2c4e3ebd202aa3dba8f1400888\": container with ID starting with 54a5d4f6e407b7fb337df89bcb7ddfdadf4d4a2c4e3ebd202aa3dba8f1400888 not found: ID does not exist"
Sep 11 16:54:24.514689 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.514650    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"]
Sep 11 16:54:24.515335 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.515317    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6594b0e2-09f2-4639-9f66-15ac7cc55b85" containerName="test-curl-7783-643e88"
Sep 11 16:54:24.515385 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.515341    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="6594b0e2-09f2-4639-9f66-15ac7cc55b85" containerName="test-curl-7783-643e88"
Sep 11 16:54:24.515385 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.515377    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy"
Sep 11 16:54:24.515449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.515386    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy"
Sep 11 16:54:24.515449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.515423    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy"
Sep 11 16:54:24.515449 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.515431    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy"
Sep 11 16:54:24.515554 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.515526    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy"
Sep 11 16:54:24.515554 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.515543    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eeafa2d-7736-4cb9-96c8-1b822270dcb4" containerName="istio-proxy"
Sep 11 16:54:24.515615 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.515555    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="6594b0e2-09f2-4639-9f66-15ac7cc55b85" containerName="test-curl-7783-643e88"
Sep 11 16:54:24.520536 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.520516    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.522984 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.522956    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ff7416\"/\"rate-test-model-ff7416-kserve-self-signed-certs\""
Sep 11 16:54:24.527387 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.527363    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"]
Sep 11 16:54:24.673750 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.673696    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w668\" (UniqueName: \"kubernetes.io/projected/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-kube-api-access-7w668\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.673982 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.673788    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-tmp-dir\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.673982 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.673828    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-home\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.673982 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.673921    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-tls-certs\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.673982 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.673945    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-model-cache\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.674121 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.674014    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-dshm\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.775033 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.774913    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w668\" (UniqueName: \"kubernetes.io/projected/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-kube-api-access-7w668\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.775033 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.775001    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-tmp-dir\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.775285 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.775051    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-home\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.775285 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.775087    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-tls-certs\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.775285 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.775116    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-model-cache\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.775285 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.775189    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-dshm\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.775502 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.775480    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-tmp-dir\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.775650 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.775622    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-home\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.775759 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.775651    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-model-cache\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.778270 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.778246    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-dshm\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.778430 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.778409    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-tls-certs\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.786935 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.786905    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w668\" (UniqueName: \"kubernetes.io/projected/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-kube-api-access-7w668\") pod \"rate-test-model-ff7416-kserve-8677c46c99-jb5jw\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") " pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:24.831800 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:24.831756    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:25.117406 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:25.117329    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"]
Sep 11 16:54:25.120790 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:54:25.120761    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d0a9fb5_f078_4ce3_bed7_1d2b22e44c36.slice/crio-d8970c73018ed7238d78613629879f90e305bda48326050a2404f0195192cb03 WatchSource:0}: Error finding container d8970c73018ed7238d78613629879f90e305bda48326050a2404f0195192cb03: Status 404 returned error can't find the container with id d8970c73018ed7238d78613629879f90e305bda48326050a2404f0195192cb03
Sep 11 16:54:25.166761 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:25.166728    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw" event={"ID":"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36","Type":"ContainerStarted","Data":"d8970c73018ed7238d78613629879f90e305bda48326050a2404f0195192cb03"}
Sep 11 16:54:26.174031 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:26.173996    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw" event={"ID":"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36","Type":"ContainerStarted","Data":"864c57bc5b22e00b7291c595263cb6c17e9f25b8bd337078ab351fd0c1318279"}
Sep 11 16:54:26.194525 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:26.194469    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw" podStartSLOduration=2.194450819 podStartE2EDuration="2.194450819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:54:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:54:26.193538377 +0000 UTC m=+1484.746708306" watchObservedRunningTime="2026-09-11 16:54:26.194450819 +0000 UTC m=+1484.747620745"
Sep 11 16:54:27.483137 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:27.483094    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78"]
Sep 11 16:54:27.489050 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:27.489017    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-16265e-zkh78"]
Sep 11 16:54:27.939110 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:27.939075    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51f8d89d-3a4c-486f-b360-ce43a9add927" path="/var/lib/kubelet/pods/51f8d89d-3a4c-486f-b360-ce43a9add927/volumes"
Sep 11 16:54:31.558585 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:31.558551    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:54:34.734378 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:34.734342    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"]
Sep 11 16:54:34.734926 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:34.734570    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj" podUID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" containerName="istio-proxy" containerID="cri-o://d50e1c20046ca417a2fea7c71066e9cb40663da89500a19b6c1ee1cde7e5e9be" gracePeriod=30
Sep 11 16:54:34.832282 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:34.832244    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:34.832547 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:34.832523    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:34.854085 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:34.854056    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:34.930190 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:34.930140    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn" podUID="10b3bf6b-25dc-4978-8ef9-a07693a3a76e" containerName="main" containerID="cri-o://decf985b4fb77836ec174fbfe0ac552310f9d623f04d7213942f0d00cc15578b" gracePeriod=45
Sep 11 16:54:35.118413 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.118381    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h"]
Sep 11 16:54:35.122877 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.122836    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h"
Sep 11 16:54:35.125497 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.125467    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:54:35.125648 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.125476    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:54:35.131713 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.131684    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h"]
Sep 11 16:54:35.189778 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.189739    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:54:35.236709 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.236673    2545 generic.go:356] "Generic (PLEG): container finished" podID="10b3bf6b-25dc-4978-8ef9-a07693a3a76e" containerID="decf985b4fb77836ec174fbfe0ac552310f9d623f04d7213942f0d00cc15578b" exitCode=0
Sep 11 16:54:35.236900 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.236731    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"
Sep 11 16:54:35.236900 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.236755    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn" event={"ID":"10b3bf6b-25dc-4978-8ef9-a07693a3a76e","Type":"ContainerDied","Data":"decf985b4fb77836ec174fbfe0ac552310f9d623f04d7213942f0d00cc15578b"}
Sep 11 16:54:35.236900 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.236806    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn" event={"ID":"10b3bf6b-25dc-4978-8ef9-a07693a3a76e","Type":"ContainerDied","Data":"978798ffe61f19f04bc098aa625a506174c87b9958f56a60630d7ad971847360"}
Sep 11 16:54:35.236900 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.236832    2545 scope.go:117] "RemoveContainer" containerID="decf985b4fb77836ec174fbfe0ac552310f9d623f04d7213942f0d00cc15578b"
Sep 11 16:54:35.246268 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.246247    2545 scope.go:117] "RemoveContainer" containerID="decf985b4fb77836ec174fbfe0ac552310f9d623f04d7213942f0d00cc15578b"
Sep 11 16:54:35.246579 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:54:35.246560    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"decf985b4fb77836ec174fbfe0ac552310f9d623f04d7213942f0d00cc15578b\": container with ID starting with decf985b4fb77836ec174fbfe0ac552310f9d623f04d7213942f0d00cc15578b not found: ID does not exist" containerID="decf985b4fb77836ec174fbfe0ac552310f9d623f04d7213942f0d00cc15578b"
Sep 11 16:54:35.246635 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.246590    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"decf985b4fb77836ec174fbfe0ac552310f9d623f04d7213942f0d00cc15578b"} err="failed to get container status \"decf985b4fb77836ec174fbfe0ac552310f9d623f04d7213942f0d00cc15578b\": rpc error: code = NotFound desc = could not find container \"decf985b4fb77836ec174fbfe0ac552310f9d623f04d7213942f0d00cc15578b\": container with ID starting with decf985b4fb77836ec174fbfe0ac552310f9d623f04d7213942f0d00cc15578b not found: ID does not exist"
Sep 11 16:54:35.252931 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.252904    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:54:35.290588 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.290553    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-699f8\" (UniqueName: \"kubernetes.io/projected/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-kube-api-access-699f8\") pod \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") "
Sep 11 16:54:35.290588 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.290606    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-tmp-dir\") pod \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") "
Sep 11 16:54:35.290854 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.290667    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-home\") pod \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") "
Sep 11 16:54:35.290854 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.290716    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-dshm\") pod \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") "
Sep 11 16:54:35.291004 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.290855    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-tls-certs\") pod \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") "
Sep 11 16:54:35.291004 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.290899    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-model-cache\") pod \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\" (UID: \"10b3bf6b-25dc-4978-8ef9-a07693a3a76e\") "
Sep 11 16:54:35.291004 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.290955    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "10b3bf6b-25dc-4978-8ef9-a07693a3a76e" (UID: "10b3bf6b-25dc-4978-8ef9-a07693a3a76e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:35.291164 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.291090    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12dbc51e-47f9-44df-bc96-ee31f5480239-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h\" (UID: \"12dbc51e-47f9-44df-bc96-ee31f5480239\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h"
Sep 11 16:54:35.291307 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.291289    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:35.291513 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.291487    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-model-cache" (OuterVolumeSpecName: "model-cache") pod "10b3bf6b-25dc-4978-8ef9-a07693a3a76e" (UID: "10b3bf6b-25dc-4978-8ef9-a07693a3a76e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:35.291598 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.291528    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-home" (OuterVolumeSpecName: "home") pod "10b3bf6b-25dc-4978-8ef9-a07693a3a76e" (UID: "10b3bf6b-25dc-4978-8ef9-a07693a3a76e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:35.293805 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.293732    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-dshm" (OuterVolumeSpecName: "dshm") pod "10b3bf6b-25dc-4978-8ef9-a07693a3a76e" (UID: "10b3bf6b-25dc-4978-8ef9-a07693a3a76e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:35.294100 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.294072    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "10b3bf6b-25dc-4978-8ef9-a07693a3a76e" (UID: "10b3bf6b-25dc-4978-8ef9-a07693a3a76e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:54:35.294220 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.294142    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-kube-api-access-699f8" (OuterVolumeSpecName: "kube-api-access-699f8") pod "10b3bf6b-25dc-4978-8ef9-a07693a3a76e" (UID: "10b3bf6b-25dc-4978-8ef9-a07693a3a76e"). InnerVolumeSpecName "kube-api-access-699f8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:54:35.392454 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.392413    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12dbc51e-47f9-44df-bc96-ee31f5480239-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h\" (UID: \"12dbc51e-47f9-44df-bc96-ee31f5480239\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h"
Sep 11 16:54:35.392672 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.392532    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:35.392672 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.392548    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:35.392672 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.392562    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:35.392672 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.392576    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-699f8\" (UniqueName: \"kubernetes.io/projected/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-kube-api-access-699f8\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:35.392672 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.392584    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10b3bf6b-25dc-4978-8ef9-a07693a3a76e-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:35.393143 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.393124    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12dbc51e-47f9-44df-bc96-ee31f5480239-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h\" (UID: \"12dbc51e-47f9-44df-bc96-ee31f5480239\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h"
Sep 11 16:54:35.439143 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.439097    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h"
Sep 11 16:54:35.564030 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.563992    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"]
Sep 11 16:54:35.573653 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.573622    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/sub-test-model-ff7416-kserve-8885c8799-zbgdn"]
Sep 11 16:54:35.726729 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.726702    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h"]
Sep 11 16:54:35.729003 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:54:35.728975    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12dbc51e_47f9_44df_bc96_ee31f5480239.slice/crio-61750ec25ec71dcb9f792def9f9b503c3fd98ab8885ab0db21d77e52235dc33a WatchSource:0}: Error finding container 61750ec25ec71dcb9f792def9f9b503c3fd98ab8885ab0db21d77e52235dc33a: Status 404 returned error can't find the container with id 61750ec25ec71dcb9f792def9f9b503c3fd98ab8885ab0db21d77e52235dc33a
Sep 11 16:54:35.886102 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.886007    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k" podUID="a2d37ed4-fd9f-48d0-9825-b1165a4b42f6" containerName="main" containerID="cri-o://0f5353b978d9c5d933927eeca2288ff3689038a7254aab4ce1e7e1d435d8f879" gracePeriod=45
Sep 11 16:54:35.938475 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:35.938435    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10b3bf6b-25dc-4978-8ef9-a07693a3a76e" path="/var/lib/kubelet/pods/10b3bf6b-25dc-4978-8ef9-a07693a3a76e/volumes"
Sep 11 16:54:36.145946 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.145879    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:54:36.243911 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.243850    2545 generic.go:356] "Generic (PLEG): container finished" podID="a2d37ed4-fd9f-48d0-9825-b1165a4b42f6" containerID="0f5353b978d9c5d933927eeca2288ff3689038a7254aab4ce1e7e1d435d8f879" exitCode=0
Sep 11 16:54:36.243911 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.243914    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"
Sep 11 16:54:36.244141 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.243942    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k" event={"ID":"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6","Type":"ContainerDied","Data":"0f5353b978d9c5d933927eeca2288ff3689038a7254aab4ce1e7e1d435d8f879"}
Sep 11 16:54:36.244141 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.243995    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k" event={"ID":"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6","Type":"ContainerDied","Data":"b3e8a37c90579642a7c2446ccf2eac735b96644b19c1fd0aaf1b73d5e411804b"}
Sep 11 16:54:36.244141 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.244015    2545 scope.go:117] "RemoveContainer" containerID="0f5353b978d9c5d933927eeca2288ff3689038a7254aab4ce1e7e1d435d8f879"
Sep 11 16:54:36.245682 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.245658    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h" event={"ID":"12dbc51e-47f9-44df-bc96-ee31f5480239","Type":"ContainerStarted","Data":"413344b295f83c1f768b6e466c009a6485d301c97d1759c66693a7dcbb970a47"}
Sep 11 16:54:36.245801 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.245703    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h" event={"ID":"12dbc51e-47f9-44df-bc96-ee31f5480239","Type":"ContainerStarted","Data":"61750ec25ec71dcb9f792def9f9b503c3fd98ab8885ab0db21d77e52235dc33a"}
Sep 11 16:54:36.254034 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.254013    2545 scope.go:117] "RemoveContainer" containerID="0f5353b978d9c5d933927eeca2288ff3689038a7254aab4ce1e7e1d435d8f879"
Sep 11 16:54:36.254382 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:54:36.254362    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f5353b978d9c5d933927eeca2288ff3689038a7254aab4ce1e7e1d435d8f879\": container with ID starting with 0f5353b978d9c5d933927eeca2288ff3689038a7254aab4ce1e7e1d435d8f879 not found: ID does not exist" containerID="0f5353b978d9c5d933927eeca2288ff3689038a7254aab4ce1e7e1d435d8f879"
Sep 11 16:54:36.254450 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.254392    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f5353b978d9c5d933927eeca2288ff3689038a7254aab4ce1e7e1d435d8f879"} err="failed to get container status \"0f5353b978d9c5d933927eeca2288ff3689038a7254aab4ce1e7e1d435d8f879\": rpc error: code = NotFound desc = could not find container \"0f5353b978d9c5d933927eeca2288ff3689038a7254aab4ce1e7e1d435d8f879\": container with ID starting with 0f5353b978d9c5d933927eeca2288ff3689038a7254aab4ce1e7e1d435d8f879 not found: ID does not exist"
Sep 11 16:54:36.266595 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.266512    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h" podStartSLOduration=1.266490363 podStartE2EDuration="1.266490363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:54:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:54:36.2656389 +0000 UTC m=+1494.818808851" watchObservedRunningTime="2026-09-11 16:54:36.266490363 +0000 UTC m=+1494.819660290"
Sep 11 16:54:36.301295 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.301255    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-home\") pod \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") "
Sep 11 16:54:36.301461 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.301305    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-tmp-dir\") pod \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") "
Sep 11 16:54:36.301461 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.301351    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-dshm\") pod \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") "
Sep 11 16:54:36.301461 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.301403    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-model-cache\") pod \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") "
Sep 11 16:54:36.301583 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.301475    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-tls-certs\") pod \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") "
Sep 11 16:54:36.301583 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.301513    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thv6z\" (UniqueName: \"kubernetes.io/projected/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-kube-api-access-thv6z\") pod \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\" (UID: \"a2d37ed4-fd9f-48d0-9825-b1165a4b42f6\") "
Sep 11 16:54:36.301680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.301595    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-home" (OuterVolumeSpecName: "home") pod "a2d37ed4-fd9f-48d0-9825-b1165a4b42f6" (UID: "a2d37ed4-fd9f-48d0-9825-b1165a4b42f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:36.301680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.301652    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a2d37ed4-fd9f-48d0-9825-b1165a4b42f6" (UID: "a2d37ed4-fd9f-48d0-9825-b1165a4b42f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:36.301795 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.301701    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-model-cache" (OuterVolumeSpecName: "model-cache") pod "a2d37ed4-fd9f-48d0-9825-b1165a4b42f6" (UID: "a2d37ed4-fd9f-48d0-9825-b1165a4b42f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:36.301915 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.301857    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:36.301981 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.301929    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:36.301981 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.301943    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:36.303815 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.303779    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a2d37ed4-fd9f-48d0-9825-b1165a4b42f6" (UID: "a2d37ed4-fd9f-48d0-9825-b1165a4b42f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:54:36.303815 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.303781    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-dshm" (OuterVolumeSpecName: "dshm") pod "a2d37ed4-fd9f-48d0-9825-b1165a4b42f6" (UID: "a2d37ed4-fd9f-48d0-9825-b1165a4b42f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:36.303815 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.303785    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-kube-api-access-thv6z" (OuterVolumeSpecName: "kube-api-access-thv6z") pod "a2d37ed4-fd9f-48d0-9825-b1165a4b42f6" (UID: "a2d37ed4-fd9f-48d0-9825-b1165a4b42f6"). InnerVolumeSpecName "kube-api-access-thv6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:54:36.403089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.402993    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:36.403089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.403026    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thv6z\" (UniqueName: \"kubernetes.io/projected/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-kube-api-access-thv6z\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:36.403089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.403041    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:36.573899 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.573850    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"]
Sep 11 16:54:36.582309 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:36.582270    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/sub-test-model-ce902a-kserve-7b58bf7767-78l7k"]
Sep 11 16:54:37.254647 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:37.254614    2545 generic.go:356] "Generic (PLEG): container finished" podID="12dbc51e-47f9-44df-bc96-ee31f5480239" containerID="413344b295f83c1f768b6e466c009a6485d301c97d1759c66693a7dcbb970a47" exitCode=0
Sep 11 16:54:37.255108 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:37.254706    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h" event={"ID":"12dbc51e-47f9-44df-bc96-ee31f5480239","Type":"ContainerDied","Data":"413344b295f83c1f768b6e466c009a6485d301c97d1759c66693a7dcbb970a47"}
Sep 11 16:54:37.938784 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:37.938739    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2d37ed4-fd9f-48d0-9825-b1165a4b42f6" path="/var/lib/kubelet/pods/a2d37ed4-fd9f-48d0-9825-b1165a4b42f6/volumes"
Sep 11 16:54:38.406096 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:38.406064    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h"
Sep 11 16:54:38.421117 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:38.421077    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12dbc51e-47f9-44df-bc96-ee31f5480239-maas-api-service-ca\") pod \"12dbc51e-47f9-44df-bc96-ee31f5480239\" (UID: \"12dbc51e-47f9-44df-bc96-ee31f5480239\") "
Sep 11 16:54:38.421431 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:38.421407    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12dbc51e-47f9-44df-bc96-ee31f5480239-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "12dbc51e-47f9-44df-bc96-ee31f5480239" (UID: "12dbc51e-47f9-44df-bc96-ee31f5480239"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:54:38.522099 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:38.522004    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12dbc51e-47f9-44df-bc96-ee31f5480239-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:39.269438 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:39.269404    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h" event={"ID":"12dbc51e-47f9-44df-bc96-ee31f5480239","Type":"ContainerDied","Data":"61750ec25ec71dcb9f792def9f9b503c3fd98ab8885ab0db21d77e52235dc33a"}
Sep 11 16:54:39.269438 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:39.269440    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h"
Sep 11 16:54:39.269656 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:39.269457    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61750ec25ec71dcb9f792def9f9b503c3fd98ab8885ab0db21d77e52235dc33a"
Sep 11 16:54:39.994146 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:39.994108    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:54:40.040827 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.040728    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84qwg\" (UniqueName: \"kubernetes.io/projected/b831ab69-7663-4c4d-bc43-d47f0393a8b6-kube-api-access-84qwg\") pod \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") "
Sep 11 16:54:40.040827 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.040790    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-token\") pod \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") "
Sep 11 16:54:40.041092 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.040843    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-podinfo\") pod \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") "
Sep 11 16:54:40.041092 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.040902    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-data\") pod \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") "
Sep 11 16:54:40.041092 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.040948    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istiod-ca-cert\") pod \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") "
Sep 11 16:54:40.041092 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.040987    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-workload-certs\") pod \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") "
Sep 11 16:54:40.041092 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.041075    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-workload-socket\") pod \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") "
Sep 11 16:54:40.041341 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.041111    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-credential-socket\") pod \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") "
Sep 11 16:54:40.041341 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.041213    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-envoy\") pod \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\" (UID: \"b831ab69-7663-4c4d-bc43-d47f0393a8b6\") "
Sep 11 16:54:40.041341 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.041258    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-data" (OuterVolumeSpecName: "istio-data") pod "b831ab69-7663-4c4d-bc43-d47f0393a8b6" (UID: "b831ab69-7663-4c4d-bc43-d47f0393a8b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:40.041488 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.041401    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b831ab69-7663-4c4d-bc43-d47f0393a8b6" (UID: "b831ab69-7663-4c4d-bc43-d47f0393a8b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:40.041626 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.041600    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b831ab69-7663-4c4d-bc43-d47f0393a8b6" (UID: "b831ab69-7663-4c4d-bc43-d47f0393a8b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:54:40.041921 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.041894    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:40.042035 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.041927    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:40.042035 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.041943    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:40.042270 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.042216    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b831ab69-7663-4c4d-bc43-d47f0393a8b6" (UID: "b831ab69-7663-4c4d-bc43-d47f0393a8b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:40.042420 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.042393    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b831ab69-7663-4c4d-bc43-d47f0393a8b6" (UID: "b831ab69-7663-4c4d-bc43-d47f0393a8b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:40.043421 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.043394    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b831ab69-7663-4c4d-bc43-d47f0393a8b6-kube-api-access-84qwg" (OuterVolumeSpecName: "kube-api-access-84qwg") pod "b831ab69-7663-4c4d-bc43-d47f0393a8b6" (UID: "b831ab69-7663-4c4d-bc43-d47f0393a8b6"). InnerVolumeSpecName "kube-api-access-84qwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:54:40.044081 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.044055    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b831ab69-7663-4c4d-bc43-d47f0393a8b6" (UID: "b831ab69-7663-4c4d-bc43-d47f0393a8b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:40.044081 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.044068    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b831ab69-7663-4c4d-bc43-d47f0393a8b6" (UID: "b831ab69-7663-4c4d-bc43-d47f0393a8b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:54:40.044188 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.044066    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-token" (OuterVolumeSpecName: "istio-token") pod "b831ab69-7663-4c4d-bc43-d47f0393a8b6" (UID: "b831ab69-7663-4c4d-bc43-d47f0393a8b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:54:40.143197 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.143158    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:40.143197 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.143193    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:40.143197 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.143202    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:40.143452 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.143210    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:40.143452 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.143220    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b831ab69-7663-4c4d-bc43-d47f0393a8b6-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:40.143452 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.143229    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84qwg\" (UniqueName: \"kubernetes.io/projected/b831ab69-7663-4c4d-bc43-d47f0393a8b6-kube-api-access-84qwg\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:40.278161 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.278122    2545 generic.go:356] "Generic (PLEG): container finished" podID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" containerID="d50e1c20046ca417a2fea7c71066e9cb40663da89500a19b6c1ee1cde7e5e9be" exitCode=0
Sep 11 16:54:40.278365 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.278180    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"
Sep 11 16:54:40.278365 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.278199    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj" event={"ID":"b831ab69-7663-4c4d-bc43-d47f0393a8b6","Type":"ContainerDied","Data":"d50e1c20046ca417a2fea7c71066e9cb40663da89500a19b6c1ee1cde7e5e9be"}
Sep 11 16:54:40.278365 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.278255    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj" event={"ID":"b831ab69-7663-4c4d-bc43-d47f0393a8b6","Type":"ContainerDied","Data":"19ae16c83739524458d2a707f2cf7a920869f8129a0a031ff99bc904601ee5e4"}
Sep 11 16:54:40.278365 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.278278    2545 scope.go:117] "RemoveContainer" containerID="d50e1c20046ca417a2fea7c71066e9cb40663da89500a19b6c1ee1cde7e5e9be"
Sep 11 16:54:40.288741 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.288716    2545 scope.go:117] "RemoveContainer" containerID="24feb6e6ba2150226c5f88c90306ac5beaf06e3a5ad7e4d4c21edb263679042e"
Sep 11 16:54:40.298374 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.298354    2545 scope.go:117] "RemoveContainer" containerID="d50e1c20046ca417a2fea7c71066e9cb40663da89500a19b6c1ee1cde7e5e9be"
Sep 11 16:54:40.298707 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:54:40.298688    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d50e1c20046ca417a2fea7c71066e9cb40663da89500a19b6c1ee1cde7e5e9be\": container with ID starting with d50e1c20046ca417a2fea7c71066e9cb40663da89500a19b6c1ee1cde7e5e9be not found: ID does not exist" containerID="d50e1c20046ca417a2fea7c71066e9cb40663da89500a19b6c1ee1cde7e5e9be"
Sep 11 16:54:40.298746 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.298718    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50e1c20046ca417a2fea7c71066e9cb40663da89500a19b6c1ee1cde7e5e9be"} err="failed to get container status \"d50e1c20046ca417a2fea7c71066e9cb40663da89500a19b6c1ee1cde7e5e9be\": rpc error: code = NotFound desc = could not find container \"d50e1c20046ca417a2fea7c71066e9cb40663da89500a19b6c1ee1cde7e5e9be\": container with ID starting with d50e1c20046ca417a2fea7c71066e9cb40663da89500a19b6c1ee1cde7e5e9be not found: ID does not exist"
Sep 11 16:54:40.298746 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.298736    2545 scope.go:117] "RemoveContainer" containerID="24feb6e6ba2150226c5f88c90306ac5beaf06e3a5ad7e4d4c21edb263679042e"
Sep 11 16:54:40.299058 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:54:40.299040    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24feb6e6ba2150226c5f88c90306ac5beaf06e3a5ad7e4d4c21edb263679042e\": container with ID starting with 24feb6e6ba2150226c5f88c90306ac5beaf06e3a5ad7e4d4c21edb263679042e not found: ID does not exist" containerID="24feb6e6ba2150226c5f88c90306ac5beaf06e3a5ad7e4d4c21edb263679042e"
Sep 11 16:54:40.299106 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.299065    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24feb6e6ba2150226c5f88c90306ac5beaf06e3a5ad7e4d4c21edb263679042e"} err="failed to get container status \"24feb6e6ba2150226c5f88c90306ac5beaf06e3a5ad7e4d4c21edb263679042e\": rpc error: code = NotFound desc = could not find container \"24feb6e6ba2150226c5f88c90306ac5beaf06e3a5ad7e4d4c21edb263679042e\": container with ID starting with 24feb6e6ba2150226c5f88c90306ac5beaf06e3a5ad7e4d4c21edb263679042e not found: ID does not exist"
Sep 11 16:54:40.325379 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.325346    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"]
Sep 11 16:54:40.327317 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:40.327296    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-16265e-openshift-default-c54c5d4d-d8nsj"]
Sep 11 16:54:41.938971 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:41.938937    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" path="/var/lib/kubelet/pods/b831ab69-7663-4c4d-bc43-d47f0393a8b6/volumes"
Sep 11 16:54:42.032261 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:42.032227    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:54:42.035877 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:42.035840    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:54:42.356581 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:42.356542    2545 scope.go:117] "RemoveContainer" containerID="a2bd52cde9a7295d3e8998d262dfe1a0bdab7592dec5796a22bf5102d1b0a2d2"
Sep 11 16:54:42.367110 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:42.367045    2545 scope.go:117] "RemoveContainer" containerID="3da24478ab05e3ecbeebcd43ee343ecd2e9e168dc3f3ad5f217ee841c713684f"
Sep 11 16:54:46.067155 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:46.067122    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h"]
Sep 11 16:54:46.071313 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:46.071283    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-347815-j995h"]
Sep 11 16:54:47.937926 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:47.937890    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12dbc51e-47f9-44df-bc96-ee31f5480239" path="/var/lib/kubelet/pods/12dbc51e-47f9-44df-bc96-ee31f5480239/volumes"
Sep 11 16:54:51.650638 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:51.650595    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:54:53.658738 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:53.658707    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"]
Sep 11 16:54:53.659208 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:53.658959    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv" podUID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" containerName="istio-proxy" containerID="cri-o://2e427845e3115672369b2e5176cb712e4054f6a4300f8c02a9ed79c7be33d6b4" gracePeriod=30
Sep 11 16:54:54.002105 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002072    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb"]
Sep 11 16:54:54.002479 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002468    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" containerName="istio-proxy"
Sep 11 16:54:54.002523 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002481    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" containerName="istio-proxy"
Sep 11 16:54:54.002523 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002493    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" containerName="istio-proxy"
Sep 11 16:54:54.002523 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002500    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" containerName="istio-proxy"
Sep 11 16:54:54.002523 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002507    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10b3bf6b-25dc-4978-8ef9-a07693a3a76e" containerName="main"
Sep 11 16:54:54.002523 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002513    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="10b3bf6b-25dc-4978-8ef9-a07693a3a76e" containerName="main"
Sep 11 16:54:54.002680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002526    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2d37ed4-fd9f-48d0-9825-b1165a4b42f6" containerName="main"
Sep 11 16:54:54.002680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002531    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2d37ed4-fd9f-48d0-9825-b1165a4b42f6" containerName="main"
Sep 11 16:54:54.002680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002541    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12dbc51e-47f9-44df-bc96-ee31f5480239" containerName="revoke-keys"
Sep 11 16:54:54.002680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002546    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="12dbc51e-47f9-44df-bc96-ee31f5480239" containerName="revoke-keys"
Sep 11 16:54:54.002680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002614    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="12dbc51e-47f9-44df-bc96-ee31f5480239" containerName="revoke-keys"
Sep 11 16:54:54.002680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002621    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2d37ed4-fd9f-48d0-9825-b1165a4b42f6" containerName="main"
Sep 11 16:54:54.002680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002627    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="10b3bf6b-25dc-4978-8ef9-a07693a3a76e" containerName="main"
Sep 11 16:54:54.002680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002637    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" containerName="istio-proxy"
Sep 11 16:54:54.002680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.002644    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="b831ab69-7663-4c4d-bc43-d47f0393a8b6" containerName="istio-proxy"
Sep 11 16:54:54.007482 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.007466    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb"
Sep 11 16:54:54.010244 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.010224    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:54:54.010510 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.010495    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:54:54.017101 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.017077    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb"]
Sep 11 16:54:54.084845 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.084813    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f930569-5def-4cc3-b2a8-a1a53d9cf116-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb\" (UID: \"8f930569-5def-4cc3-b2a8-a1a53d9cf116\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb"
Sep 11 16:54:54.186063 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.186030    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f930569-5def-4cc3-b2a8-a1a53d9cf116-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb\" (UID: \"8f930569-5def-4cc3-b2a8-a1a53d9cf116\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb"
Sep 11 16:54:54.186652 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.186623    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f930569-5def-4cc3-b2a8-a1a53d9cf116-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb\" (UID: \"8f930569-5def-4cc3-b2a8-a1a53d9cf116\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb"
Sep 11 16:54:54.319648 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.319553    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb"
Sep 11 16:54:54.594896 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:54.594847    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb"]
Sep 11 16:54:54.596182 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:54:54.596157    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f930569_5def_4cc3_b2a8_a1a53d9cf116.slice/crio-263bcc34543abc695414a298dae7c2d27a3d3ea6d91afc86b96850e46fa6ffec WatchSource:0}: Error finding container 263bcc34543abc695414a298dae7c2d27a3d3ea6d91afc86b96850e46fa6ffec: Status 404 returned error can't find the container with id 263bcc34543abc695414a298dae7c2d27a3d3ea6d91afc86b96850e46fa6ffec
Sep 11 16:54:55.369764 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:55.369727    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb" event={"ID":"8f930569-5def-4cc3-b2a8-a1a53d9cf116","Type":"ContainerStarted","Data":"2129f298244a90bcbf4f0b7e2c93a1eb7a4f018668f2d23d51883541d5f7e0d4"}
Sep 11 16:54:55.370224 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:55.369813    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb" event={"ID":"8f930569-5def-4cc3-b2a8-a1a53d9cf116","Type":"ContainerStarted","Data":"263bcc34543abc695414a298dae7c2d27a3d3ea6d91afc86b96850e46fa6ffec"}
Sep 11 16:54:55.390533 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:55.390477    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb" podStartSLOduration=2.390458511 podStartE2EDuration="2.390458511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:54:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:54:55.388391339 +0000 UTC m=+1513.941561265" watchObservedRunningTime="2026-09-11 16:54:55.390458511 +0000 UTC m=+1513.943628438"
Sep 11 16:54:56.377939 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:56.377899    2545 generic.go:356] "Generic (PLEG): container finished" podID="8f930569-5def-4cc3-b2a8-a1a53d9cf116" containerID="2129f298244a90bcbf4f0b7e2c93a1eb7a4f018668f2d23d51883541d5f7e0d4" exitCode=0
Sep 11 16:54:56.378331 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:56.377986    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb" event={"ID":"8f930569-5def-4cc3-b2a8-a1a53d9cf116","Type":"ContainerDied","Data":"2129f298244a90bcbf4f0b7e2c93a1eb7a4f018668f2d23d51883541d5f7e0d4"}
Sep 11 16:54:57.519522 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:57.519496    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb"
Sep 11 16:54:57.617182 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:57.617147    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f930569-5def-4cc3-b2a8-a1a53d9cf116-maas-api-service-ca\") pod \"8f930569-5def-4cc3-b2a8-a1a53d9cf116\" (UID: \"8f930569-5def-4cc3-b2a8-a1a53d9cf116\") "
Sep 11 16:54:57.617483 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:57.617463    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f930569-5def-4cc3-b2a8-a1a53d9cf116-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8f930569-5def-4cc3-b2a8-a1a53d9cf116" (UID: "8f930569-5def-4cc3-b2a8-a1a53d9cf116"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:54:57.718368 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:57.718273    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f930569-5def-4cc3-b2a8-a1a53d9cf116-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:58.390561 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:58.390524    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb" event={"ID":"8f930569-5def-4cc3-b2a8-a1a53d9cf116","Type":"ContainerDied","Data":"263bcc34543abc695414a298dae7c2d27a3d3ea6d91afc86b96850e46fa6ffec"}
Sep 11 16:54:58.390561 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:58.390554    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb"
Sep 11 16:54:58.390561 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:58.390572    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="263bcc34543abc695414a298dae7c2d27a3d3ea6d91afc86b96850e46fa6ffec"
Sep 11 16:54:58.914619 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:58.914594    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:54:59.029448 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.029418    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-envoy\") pod \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") "
Sep 11 16:54:59.029602 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.029455    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-data\") pod \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") "
Sep 11 16:54:59.029602 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.029480    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-token\") pod \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") "
Sep 11 16:54:59.029602 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.029534    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-workload-socket\") pod \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") "
Sep 11 16:54:59.029602 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.029563    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-workload-certs\") pod \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") "
Sep 11 16:54:59.029826 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.029631    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-podinfo\") pod \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") "
Sep 11 16:54:59.029826 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.029673    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj6jd\" (UniqueName: \"kubernetes.io/projected/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-kube-api-access-vj6jd\") pod \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") "
Sep 11 16:54:59.029826 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.029726    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-data" (OuterVolumeSpecName: "istio-data") pod "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" (UID: "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:59.029826 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.029750    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-credential-socket\") pod \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") "
Sep 11 16:54:59.030090 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.029854    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istiod-ca-cert\") pod \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\" (UID: \"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6\") "
Sep 11 16:54:59.030090 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.029850    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" (UID: "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:59.030090 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.029935    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" (UID: "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:59.030090 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.030033    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" (UID: "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:59.030463 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.030284    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" (UID: "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:54:59.030463 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.030317    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:59.030581 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.030488    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:59.030581 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.030504    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:59.030581 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.030518    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:59.031905 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.031841    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" (UID: "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:54:59.032002 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.031910    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" (UID: "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:54:59.032002 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.031954    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-token" (OuterVolumeSpecName: "istio-token") pod "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" (UID: "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:54:59.032311 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.032077    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-kube-api-access-vj6jd" (OuterVolumeSpecName: "kube-api-access-vj6jd") pod "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" (UID: "a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6"). InnerVolumeSpecName "kube-api-access-vj6jd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:54:59.131680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.131655    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:59.131680 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.131677    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj6jd\" (UniqueName: \"kubernetes.io/projected/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-kube-api-access-vj6jd\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:59.131826 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.131687    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:59.131826 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.131697    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:59.131826 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.131706    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:54:59.398161 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.398072    2545 generic.go:356] "Generic (PLEG): container finished" podID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" containerID="2e427845e3115672369b2e5176cb712e4054f6a4300f8c02a9ed79c7be33d6b4" exitCode=0
Sep 11 16:54:59.398161 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.398124    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"
Sep 11 16:54:59.398161 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.398138    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv" event={"ID":"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6","Type":"ContainerDied","Data":"2e427845e3115672369b2e5176cb712e4054f6a4300f8c02a9ed79c7be33d6b4"}
Sep 11 16:54:59.398404 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.398183    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv" event={"ID":"a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6","Type":"ContainerDied","Data":"a4796bb5d0c781a3454f3a2f372534dceb193485a0f0ccfb9ed3f1d03fb425ff"}
Sep 11 16:54:59.398404 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.398206    2545 scope.go:117] "RemoveContainer" containerID="2e427845e3115672369b2e5176cb712e4054f6a4300f8c02a9ed79c7be33d6b4"
Sep 11 16:54:59.408620 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.408600    2545 scope.go:117] "RemoveContainer" containerID="928d993fbb4507ce311b1af3a9dd7ffd48feb6ae5a08ba9b6648a6fb5b03420b"
Sep 11 16:54:59.417337 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.417311    2545 scope.go:117] "RemoveContainer" containerID="2e427845e3115672369b2e5176cb712e4054f6a4300f8c02a9ed79c7be33d6b4"
Sep 11 16:54:59.417626 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:54:59.417609    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e427845e3115672369b2e5176cb712e4054f6a4300f8c02a9ed79c7be33d6b4\": container with ID starting with 2e427845e3115672369b2e5176cb712e4054f6a4300f8c02a9ed79c7be33d6b4 not found: ID does not exist" containerID="2e427845e3115672369b2e5176cb712e4054f6a4300f8c02a9ed79c7be33d6b4"
Sep 11 16:54:59.417675 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.417638    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e427845e3115672369b2e5176cb712e4054f6a4300f8c02a9ed79c7be33d6b4"} err="failed to get container status \"2e427845e3115672369b2e5176cb712e4054f6a4300f8c02a9ed79c7be33d6b4\": rpc error: code = NotFound desc = could not find container \"2e427845e3115672369b2e5176cb712e4054f6a4300f8c02a9ed79c7be33d6b4\": container with ID starting with 2e427845e3115672369b2e5176cb712e4054f6a4300f8c02a9ed79c7be33d6b4 not found: ID does not exist"
Sep 11 16:54:59.417675 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.417655    2545 scope.go:117] "RemoveContainer" containerID="928d993fbb4507ce311b1af3a9dd7ffd48feb6ae5a08ba9b6648a6fb5b03420b"
Sep 11 16:54:59.417956 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:54:59.417937    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"928d993fbb4507ce311b1af3a9dd7ffd48feb6ae5a08ba9b6648a6fb5b03420b\": container with ID starting with 928d993fbb4507ce311b1af3a9dd7ffd48feb6ae5a08ba9b6648a6fb5b03420b not found: ID does not exist" containerID="928d993fbb4507ce311b1af3a9dd7ffd48feb6ae5a08ba9b6648a6fb5b03420b"
Sep 11 16:54:59.418004 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.417963    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"928d993fbb4507ce311b1af3a9dd7ffd48feb6ae5a08ba9b6648a6fb5b03420b"} err="failed to get container status \"928d993fbb4507ce311b1af3a9dd7ffd48feb6ae5a08ba9b6648a6fb5b03420b\": rpc error: code = NotFound desc = could not find container \"928d993fbb4507ce311b1af3a9dd7ffd48feb6ae5a08ba9b6648a6fb5b03420b\": container with ID starting with 928d993fbb4507ce311b1af3a9dd7ffd48feb6ae5a08ba9b6648a6fb5b03420b not found: ID does not exist"
Sep 11 16:54:59.423809 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.423782    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"]
Sep 11 16:54:59.431649 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.431593    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-347815-openshift-default-858997b89d-295hv"]
Sep 11 16:54:59.939250 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:54:59.939193    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" path="/var/lib/kubelet/pods/a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6/volumes"
Sep 11 16:55:05.201329 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:05.201222    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb"]
Sep 11 16:55:05.204048 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:05.204021    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3e9c29-vf8nb"]
Sep 11 16:55:05.938933 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:05.938892    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f930569-5def-4cc3-b2a8-a1a53d9cf116" path="/var/lib/kubelet/pods/8f930569-5def-4cc3-b2a8-a1a53d9cf116/volumes"
Sep 11 16:55:08.265470 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:08.265435    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:55:10.617958 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.617924    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"]
Sep 11 16:55:10.618361 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.618303    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" containerName="istio-proxy"
Sep 11 16:55:10.618361 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.618314    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" containerName="istio-proxy"
Sep 11 16:55:10.618361 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.618323    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f930569-5def-4cc3-b2a8-a1a53d9cf116" containerName="revoke-keys"
Sep 11 16:55:10.618361 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.618329    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f930569-5def-4cc3-b2a8-a1a53d9cf116" containerName="revoke-keys"
Sep 11 16:55:10.618361 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.618340    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" containerName="istio-proxy"
Sep 11 16:55:10.618361 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.618346    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" containerName="istio-proxy"
Sep 11 16:55:10.618589 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.618414    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" containerName="istio-proxy"
Sep 11 16:55:10.618589 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.618424    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8bc8b91-64bc-48bc-b3c4-0d86e1f3cfb6" containerName="istio-proxy"
Sep 11 16:55:10.618589 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.618430    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f930569-5def-4cc3-b2a8-a1a53d9cf116" containerName="revoke-keys"
Sep 11 16:55:10.623141 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.623125    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.625929 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.625908    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-ce902a\"/\"rate-test-model-ce902a-kserve-self-signed-certs\""
Sep 11 16:55:10.625929 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.625926    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-ce902a\"/\"openshift-service-ca.crt\""
Sep 11 16:55:10.626117 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.626029    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-ce902a\"/\"default-dockercfg-vj4dp\""
Sep 11 16:55:10.626117 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.626064    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-ce902a\"/\"kube-root-ca.crt\""
Sep 11 16:55:10.633645 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.633625    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"]
Sep 11 16:55:10.745592 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.745559    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-dshm\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.745592 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.745594    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k95f5\" (UniqueName: \"kubernetes.io/projected/bf1da21d-7de6-4f0e-9341-f4add2840c0a-kube-api-access-k95f5\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.745791 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.745620    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-tmp-dir\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.745791 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.745683    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-home\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.745791 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.745714    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf1da21d-7de6-4f0e-9341-f4add2840c0a-tls-certs\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.745941 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.745806    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-model-cache\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.846649 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.846613    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-dshm\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.846649 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.846653    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k95f5\" (UniqueName: \"kubernetes.io/projected/bf1da21d-7de6-4f0e-9341-f4add2840c0a-kube-api-access-k95f5\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.846932 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.846683    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-tmp-dir\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.846932 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.846798    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-home\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.846932 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.846829    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf1da21d-7de6-4f0e-9341-f4add2840c0a-tls-certs\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.846932 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.846920    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-model-cache\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.847136 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.847093    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-tmp-dir\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.847197 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.847169    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-home\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.847249 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.847208    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-model-cache\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.849686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.849655    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-dshm\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.850040 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.850017    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf1da21d-7de6-4f0e-9341-f4add2840c0a-tls-certs\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.865082 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.865059    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k95f5\" (UniqueName: \"kubernetes.io/projected/bf1da21d-7de6-4f0e-9341-f4add2840c0a-kube-api-access-k95f5\") pod \"rate-test-model-ce902a-kserve-689475fb9f-zg5zk\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") " pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:10.932409 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:10.932345    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:11.229466 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:11.229433    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"]
Sep 11 16:55:11.230245 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:55:11.230223    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf1da21d_7de6_4f0e_9341_f4add2840c0a.slice/crio-caa24f446ded224c55a5fa8d0cd44e98d2c5d6b081493b5137b057e1428c6bc8 WatchSource:0}: Error finding container caa24f446ded224c55a5fa8d0cd44e98d2c5d6b081493b5137b057e1428c6bc8: Status 404 returned error can't find the container with id caa24f446ded224c55a5fa8d0cd44e98d2c5d6b081493b5137b057e1428c6bc8
Sep 11 16:55:11.465147 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:11.465110    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk" event={"ID":"bf1da21d-7de6-4f0e-9341-f4add2840c0a","Type":"ContainerStarted","Data":"ad75df29f13bfd107fd826704a03462ad5c85b51771e71e1e3164ef69aece150"}
Sep 11 16:55:11.465315 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:11.465160    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk" event={"ID":"bf1da21d-7de6-4f0e-9341-f4add2840c0a","Type":"ContainerStarted","Data":"caa24f446ded224c55a5fa8d0cd44e98d2c5d6b081493b5137b057e1428c6bc8"}
Sep 11 16:55:11.501520 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:11.501482    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk" podStartSLOduration=1.501468129 podStartE2EDuration="1.501468129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:55:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:55:11.501070941 +0000 UTC m=+1530.054240868" watchObservedRunningTime="2026-09-11 16:55:11.501468129 +0000 UTC m=+1530.054638055"
Sep 11 16:55:13.760082 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:13.760045    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"]
Sep 11 16:55:13.760526 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:13.760250    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6" podUID="62c50559-360b-451a-afe0-7ac96f1c3935" containerName="istio-proxy" containerID="cri-o://228cc4ee3e0b24f3b65b5d32759746298b4bb5c83d958aa30a5eab30aa0f3f77" gracePeriod=30
Sep 11 16:55:19.031019 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.030958    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:55:19.121621 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.121590    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62c50559-360b-451a-afe0-7ac96f1c3935-istiod-ca-cert\") pod \"62c50559-360b-451a-afe0-7ac96f1c3935\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") "
Sep 11 16:55:19.121762 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.121637    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jm68d\" (UniqueName: \"kubernetes.io/projected/62c50559-360b-451a-afe0-7ac96f1c3935-kube-api-access-jm68d\") pod \"62c50559-360b-451a-afe0-7ac96f1c3935\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") "
Sep 11 16:55:19.121762 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.121686    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-workload-socket\") pod \"62c50559-360b-451a-afe0-7ac96f1c3935\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") "
Sep 11 16:55:19.121762 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.121733    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-istio-data\") pod \"62c50559-360b-451a-afe0-7ac96f1c3935\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") "
Sep 11 16:55:19.121940 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.121762    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-istio-envoy\") pod \"62c50559-360b-451a-afe0-7ac96f1c3935\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") "
Sep 11 16:55:19.121940 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.121801    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62c50559-360b-451a-afe0-7ac96f1c3935-istio-token\") pod \"62c50559-360b-451a-afe0-7ac96f1c3935\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") "
Sep 11 16:55:19.121940 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.121851    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-credential-socket\") pod \"62c50559-360b-451a-afe0-7ac96f1c3935\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") "
Sep 11 16:55:19.121940 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.121921    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62c50559-360b-451a-afe0-7ac96f1c3935-istio-podinfo\") pod \"62c50559-360b-451a-afe0-7ac96f1c3935\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") "
Sep 11 16:55:19.122146 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.121954    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-workload-certs\") pod \"62c50559-360b-451a-afe0-7ac96f1c3935\" (UID: \"62c50559-360b-451a-afe0-7ac96f1c3935\") "
Sep 11 16:55:19.122146 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.122034    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "62c50559-360b-451a-afe0-7ac96f1c3935" (UID: "62c50559-360b-451a-afe0-7ac96f1c3935"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:55:19.122146 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.122126    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "62c50559-360b-451a-afe0-7ac96f1c3935" (UID: "62c50559-360b-451a-afe0-7ac96f1c3935"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:55:19.122146 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.122133    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62c50559-360b-451a-afe0-7ac96f1c3935-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "62c50559-360b-451a-afe0-7ac96f1c3935" (UID: "62c50559-360b-451a-afe0-7ac96f1c3935"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:55:19.122361 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.122309    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "62c50559-360b-451a-afe0-7ac96f1c3935" (UID: "62c50559-360b-451a-afe0-7ac96f1c3935"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:55:19.122511 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.122437    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:55:19.122511 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.122461    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:55:19.122511 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.122476    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:55:19.122511 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.122468    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-istio-data" (OuterVolumeSpecName: "istio-data") pod "62c50559-360b-451a-afe0-7ac96f1c3935" (UID: "62c50559-360b-451a-afe0-7ac96f1c3935"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:55:19.122511 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.122489    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62c50559-360b-451a-afe0-7ac96f1c3935-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:55:19.124093 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.124071    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62c50559-360b-451a-afe0-7ac96f1c3935-istio-token" (OuterVolumeSpecName: "istio-token") pod "62c50559-360b-451a-afe0-7ac96f1c3935" (UID: "62c50559-360b-451a-afe0-7ac96f1c3935"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:55:19.124182 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.124089    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62c50559-360b-451a-afe0-7ac96f1c3935-kube-api-access-jm68d" (OuterVolumeSpecName: "kube-api-access-jm68d") pod "62c50559-360b-451a-afe0-7ac96f1c3935" (UID: "62c50559-360b-451a-afe0-7ac96f1c3935"). InnerVolumeSpecName "kube-api-access-jm68d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:55:19.124182 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.124116    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62c50559-360b-451a-afe0-7ac96f1c3935-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "62c50559-360b-451a-afe0-7ac96f1c3935" (UID: "62c50559-360b-451a-afe0-7ac96f1c3935"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:55:19.124356 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.124342    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "62c50559-360b-451a-afe0-7ac96f1c3935" (UID: "62c50559-360b-451a-afe0-7ac96f1c3935"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:55:19.223046 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.223026    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jm68d\" (UniqueName: \"kubernetes.io/projected/62c50559-360b-451a-afe0-7ac96f1c3935-kube-api-access-jm68d\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:55:19.223046 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.223045    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:55:19.223179 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.223055    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62c50559-360b-451a-afe0-7ac96f1c3935-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:55:19.223179 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.223062    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62c50559-360b-451a-afe0-7ac96f1c3935-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:55:19.223179 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.223070    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62c50559-360b-451a-afe0-7ac96f1c3935-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:55:19.520834 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.520803    2545 generic.go:356] "Generic (PLEG): container finished" podID="62c50559-360b-451a-afe0-7ac96f1c3935" containerID="228cc4ee3e0b24f3b65b5d32759746298b4bb5c83d958aa30a5eab30aa0f3f77" exitCode=0
Sep 11 16:55:19.521055 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.520854    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"
Sep 11 16:55:19.521055 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.520890    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6" event={"ID":"62c50559-360b-451a-afe0-7ac96f1c3935","Type":"ContainerDied","Data":"228cc4ee3e0b24f3b65b5d32759746298b4bb5c83d958aa30a5eab30aa0f3f77"}
Sep 11 16:55:19.521055 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.520949    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6" event={"ID":"62c50559-360b-451a-afe0-7ac96f1c3935","Type":"ContainerDied","Data":"892dffe6bc213d358ce14c40160b7275d8377f34fb42ef369c371c065391eb6d"}
Sep 11 16:55:19.521055 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.520977    2545 scope.go:117] "RemoveContainer" containerID="228cc4ee3e0b24f3b65b5d32759746298b4bb5c83d958aa30a5eab30aa0f3f77"
Sep 11 16:55:19.531085 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.531068    2545 scope.go:117] "RemoveContainer" containerID="d8fb7bf6e972828264239641eab52b4b349271ded3d02efa2522e1ebe544aeec"
Sep 11 16:55:19.539464 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.539447    2545 scope.go:117] "RemoveContainer" containerID="228cc4ee3e0b24f3b65b5d32759746298b4bb5c83d958aa30a5eab30aa0f3f77"
Sep 11 16:55:19.539733 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:55:19.539713    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"228cc4ee3e0b24f3b65b5d32759746298b4bb5c83d958aa30a5eab30aa0f3f77\": container with ID starting with 228cc4ee3e0b24f3b65b5d32759746298b4bb5c83d958aa30a5eab30aa0f3f77 not found: ID does not exist" containerID="228cc4ee3e0b24f3b65b5d32759746298b4bb5c83d958aa30a5eab30aa0f3f77"
Sep 11 16:55:19.539818 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.539740    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"228cc4ee3e0b24f3b65b5d32759746298b4bb5c83d958aa30a5eab30aa0f3f77"} err="failed to get container status \"228cc4ee3e0b24f3b65b5d32759746298b4bb5c83d958aa30a5eab30aa0f3f77\": rpc error: code = NotFound desc = could not find container \"228cc4ee3e0b24f3b65b5d32759746298b4bb5c83d958aa30a5eab30aa0f3f77\": container with ID starting with 228cc4ee3e0b24f3b65b5d32759746298b4bb5c83d958aa30a5eab30aa0f3f77 not found: ID does not exist"
Sep 11 16:55:19.539818 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.539756    2545 scope.go:117] "RemoveContainer" containerID="d8fb7bf6e972828264239641eab52b4b349271ded3d02efa2522e1ebe544aeec"
Sep 11 16:55:19.540065 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:55:19.540050    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8fb7bf6e972828264239641eab52b4b349271ded3d02efa2522e1ebe544aeec\": container with ID starting with d8fb7bf6e972828264239641eab52b4b349271ded3d02efa2522e1ebe544aeec not found: ID does not exist" containerID="d8fb7bf6e972828264239641eab52b4b349271ded3d02efa2522e1ebe544aeec"
Sep 11 16:55:19.540117 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.540069    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8fb7bf6e972828264239641eab52b4b349271ded3d02efa2522e1ebe544aeec"} err="failed to get container status \"d8fb7bf6e972828264239641eab52b4b349271ded3d02efa2522e1ebe544aeec\": rpc error: code = NotFound desc = could not find container \"d8fb7bf6e972828264239641eab52b4b349271ded3d02efa2522e1ebe544aeec\": container with ID starting with d8fb7bf6e972828264239641eab52b4b349271ded3d02efa2522e1ebe544aeec not found: ID does not exist"
Sep 11 16:55:19.551996 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.551966    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"]
Sep 11 16:55:19.555562 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.555543    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-3e9c29-openshift-default-8588d7f646-dhzx6"]
Sep 11 16:55:19.938553 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:19.938477    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62c50559-360b-451a-afe0-7ac96f1c3935" path="/var/lib/kubelet/pods/62c50559-360b-451a-afe0-7ac96f1c3935/volumes"
Sep 11 16:55:20.933428 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:20.933398    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:20.933829 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:20.933583    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:20.946464 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:20.946436    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:21.545198 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:21.545169    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:55:21.840502 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:21.840411    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:55:34.754094 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:34.754039    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:55:44.421273 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:44.421232    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:55:57.347581 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:55:57.347544    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:56:01.661818 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:01.661783    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"]
Sep 11 16:56:02.714244 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:02.714202    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"]
Sep 11 16:56:03.535123 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.535088    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"]
Sep 11 16:56:03.535554 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.535535    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62c50559-360b-451a-afe0-7ac96f1c3935" containerName="istio-proxy"
Sep 11 16:56:03.535554 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.535555    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="62c50559-360b-451a-afe0-7ac96f1c3935" containerName="istio-proxy"
Sep 11 16:56:03.535689 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.535570    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62c50559-360b-451a-afe0-7ac96f1c3935" containerName="istio-proxy"
Sep 11 16:56:03.535689 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.535576    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="62c50559-360b-451a-afe0-7ac96f1c3935" containerName="istio-proxy"
Sep 11 16:56:03.535689 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.535655    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="62c50559-360b-451a-afe0-7ac96f1c3935" containerName="istio-proxy"
Sep 11 16:56:03.540438 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.540339    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.544808 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.544779    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-519b89-openshift-default-dockercfg-ncfzx\""
Sep 11 16:56:03.558095 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.558061    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"]
Sep 11 16:56:03.576633 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.576583    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-istio-envoy\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.576840 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.576644    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-istio-data\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.576840 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.576682    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0508d476-76f3-4dd0-b873-7392d519a751-istiod-ca-cert\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.576840 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.576715    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzh6x\" (UniqueName: \"kubernetes.io/projected/0508d476-76f3-4dd0-b873-7392d519a751-kube-api-access-wzh6x\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.576840 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.576819    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0508d476-76f3-4dd0-b873-7392d519a751-istio-podinfo\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.577074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.576855    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-credential-socket\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.577074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.576935    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0508d476-76f3-4dd0-b873-7392d519a751-istio-token\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.577074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.576991    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-workload-socket\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.577074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.577041    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-workload-certs\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.678357 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.678322    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-workload-socket\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.678576 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.678377    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-workload-certs\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.678576 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.678418    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-istio-envoy\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.678576 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.678440    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-istio-data\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.678576 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.678465    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0508d476-76f3-4dd0-b873-7392d519a751-istiod-ca-cert\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.678576 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.678495    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzh6x\" (UniqueName: \"kubernetes.io/projected/0508d476-76f3-4dd0-b873-7392d519a751-kube-api-access-wzh6x\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.678576 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.678576    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0508d476-76f3-4dd0-b873-7392d519a751-istio-podinfo\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.678937 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.678601    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-credential-socket\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.678937 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.678650    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0508d476-76f3-4dd0-b873-7392d519a751-istio-token\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.678937 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.678829    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-workload-socket\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.679106 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.679067    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-istio-data\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.679260 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.679215    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-workload-certs\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.679528 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.679491    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0508d476-76f3-4dd0-b873-7392d519a751-istiod-ca-cert\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.679760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.679735    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-credential-socket\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.681150 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.681132    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-istio-envoy\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.681740 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.681721    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0508d476-76f3-4dd0-b873-7392d519a751-istio-podinfo\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.689297 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.689261    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzh6x\" (UniqueName: \"kubernetes.io/projected/0508d476-76f3-4dd0-b873-7392d519a751-kube-api-access-wzh6x\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.689993 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.689971    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0508d476-76f3-4dd0-b873-7392d519a751-istio-token\") pod \"e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") " pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:03.853504 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:03.853397    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:04.185211 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:04.185179    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"]
Sep 11 16:56:04.187266 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:56:04.187237    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0508d476_76f3_4dd0_b873_7392d519a751.slice/crio-e366cfdf416642f8adb75a258a2e0eb256800ccf5be6e9973bf42989460deaf8 WatchSource:0}: Error finding container e366cfdf416642f8adb75a258a2e0eb256800ccf5be6e9973bf42989460deaf8: Status 404 returned error can't find the container with id e366cfdf416642f8adb75a258a2e0eb256800ccf5be6e9973bf42989460deaf8
Sep 11 16:56:04.190027 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:04.189997    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:56:04.190117 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:04.190062    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:56:04.190117 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:04.190094    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:56:04.779821 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:04.779780    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" event={"ID":"0508d476-76f3-4dd0-b873-7392d519a751","Type":"ContainerStarted","Data":"dc67b3bf036c525149d2e999ddc2ea8e5b44af9198c3109bdebcc4734a9bef42"}
Sep 11 16:56:04.780035 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:04.779845    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" event={"ID":"0508d476-76f3-4dd0-b873-7392d519a751","Type":"ContainerStarted","Data":"e366cfdf416642f8adb75a258a2e0eb256800ccf5be6e9973bf42989460deaf8"}
Sep 11 16:56:04.811250 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:04.811187    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" podStartSLOduration=1.8111674199999999 podStartE2EDuration="1.81116742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:56:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:56:04.808075741 +0000 UTC m=+1583.361245690" watchObservedRunningTime="2026-09-11 16:56:04.81116742 +0000 UTC m=+1583.364337346"
Sep 11 16:56:04.853910 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:04.853840    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:05.860966 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:05.860921    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:05.950732 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:05.950695    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:56:06.790945 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:06.790906    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:06.792259 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:06.792232    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:16.699700 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:16.699595    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw" podUID="6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36" containerName="main" containerID="cri-o://864c57bc5b22e00b7291c595263cb6c17e9f25b8bd337078ab351fd0c1318279" gracePeriod=45
Sep 11 16:56:16.852529 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:16.852494    2545 generic.go:356] "Generic (PLEG): container finished" podID="6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36" containerID="864c57bc5b22e00b7291c595263cb6c17e9f25b8bd337078ab351fd0c1318279" exitCode=0
Sep 11 16:56:16.852689 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:16.852565    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw" event={"ID":"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36","Type":"ContainerDied","Data":"864c57bc5b22e00b7291c595263cb6c17e9f25b8bd337078ab351fd0c1318279"}
Sep 11 16:56:16.974340 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:16.974313    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:56:17.130376 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.130342    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-tls-certs\") pod \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") "
Sep 11 16:56:17.130547 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.130399    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-tmp-dir\") pod \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") "
Sep 11 16:56:17.130547 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.130436    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-dshm\") pod \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") "
Sep 11 16:56:17.130547 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.130532    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-model-cache\") pod \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") "
Sep 11 16:56:17.130715 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.130572    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w668\" (UniqueName: \"kubernetes.io/projected/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-kube-api-access-7w668\") pod \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") "
Sep 11 16:56:17.130715 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.130600    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-home\") pod \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\" (UID: \"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36\") "
Sep 11 16:56:17.130715 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.130633    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36" (UID: "6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:56:17.130906 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.130778    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-model-cache" (OuterVolumeSpecName: "model-cache") pod "6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36" (UID: "6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:56:17.130906 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.130803    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-home" (OuterVolumeSpecName: "home") pod "6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36" (UID: "6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:56:17.131268 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.131247    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:56:17.131318 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.131274    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:56:17.131318 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.131289    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:56:17.132662 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.132636    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-dshm" (OuterVolumeSpecName: "dshm") pod "6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36" (UID: "6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:56:17.132779 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.132685    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36" (UID: "6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:56:17.132779 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.132691    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-kube-api-access-7w668" (OuterVolumeSpecName: "kube-api-access-7w668") pod "6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36" (UID: "6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36"). InnerVolumeSpecName "kube-api-access-7w668". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:56:17.232490 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.232411    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:56:17.232490 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.232436    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:56:17.232490 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.232445    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w668\" (UniqueName: \"kubernetes.io/projected/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36-kube-api-access-7w668\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:56:17.737695 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.737631    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk" podUID="bf1da21d-7de6-4f0e-9341-f4add2840c0a" containerName="main" containerID="cri-o://ad75df29f13bfd107fd826704a03462ad5c85b51771e71e1e3164ef69aece150" gracePeriod=45
Sep 11 16:56:17.860421 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.860389    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw" event={"ID":"6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36","Type":"ContainerDied","Data":"d8970c73018ed7238d78613629879f90e305bda48326050a2404f0195192cb03"}
Sep 11 16:56:17.860552 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.860449    2545 scope.go:117] "RemoveContainer" containerID="864c57bc5b22e00b7291c595263cb6c17e9f25b8bd337078ab351fd0c1318279"
Sep 11 16:56:17.860552 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.860392    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"
Sep 11 16:56:17.867773 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.867744    2545 generic.go:356] "Generic (PLEG): container finished" podID="bf1da21d-7de6-4f0e-9341-f4add2840c0a" containerID="ad75df29f13bfd107fd826704a03462ad5c85b51771e71e1e3164ef69aece150" exitCode=0
Sep 11 16:56:17.867884 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.867845    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk" event={"ID":"bf1da21d-7de6-4f0e-9341-f4add2840c0a","Type":"ContainerDied","Data":"ad75df29f13bfd107fd826704a03462ad5c85b51771e71e1e3164ef69aece150"}
Sep 11 16:56:17.906500 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.906462    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"]
Sep 11 16:56:17.907955 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.907923    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ff7416/rate-test-model-ff7416-kserve-8677c46c99-jb5jw"]
Sep 11 16:56:17.939245 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:17.938884    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36" path="/var/lib/kubelet/pods/6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36/volumes"
Sep 11 16:56:18.046393 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.046371    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:56:18.141483 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.141451    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k95f5\" (UniqueName: \"kubernetes.io/projected/bf1da21d-7de6-4f0e-9341-f4add2840c0a-kube-api-access-k95f5\") pod \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") "
Sep 11 16:56:18.141483 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.141487    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf1da21d-7de6-4f0e-9341-f4add2840c0a-tls-certs\") pod \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") "
Sep 11 16:56:18.141721 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.141518    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-dshm\") pod \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") "
Sep 11 16:56:18.141721 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.141578    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-tmp-dir\") pod \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") "
Sep 11 16:56:18.141721 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.141605    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-model-cache\") pod \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") "
Sep 11 16:56:18.141721 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.141689    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-home\") pod \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\" (UID: \"bf1da21d-7de6-4f0e-9341-f4add2840c0a\") "
Sep 11 16:56:18.141987 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.141856    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bf1da21d-7de6-4f0e-9341-f4add2840c0a" (UID: "bf1da21d-7de6-4f0e-9341-f4add2840c0a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:56:18.142066 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.142037    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-model-cache" (OuterVolumeSpecName: "model-cache") pod "bf1da21d-7de6-4f0e-9341-f4add2840c0a" (UID: "bf1da21d-7de6-4f0e-9341-f4add2840c0a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:56:18.142122 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.142056    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-home" (OuterVolumeSpecName: "home") pod "bf1da21d-7de6-4f0e-9341-f4add2840c0a" (UID: "bf1da21d-7de6-4f0e-9341-f4add2840c0a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:56:18.142170 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.142133    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:56:18.143780 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.143761    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-dshm" (OuterVolumeSpecName: "dshm") pod "bf1da21d-7de6-4f0e-9341-f4add2840c0a" (UID: "bf1da21d-7de6-4f0e-9341-f4add2840c0a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:56:18.144414 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.144381    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf1da21d-7de6-4f0e-9341-f4add2840c0a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf1da21d-7de6-4f0e-9341-f4add2840c0a" (UID: "bf1da21d-7de6-4f0e-9341-f4add2840c0a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:56:18.144414 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.144387    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf1da21d-7de6-4f0e-9341-f4add2840c0a-kube-api-access-k95f5" (OuterVolumeSpecName: "kube-api-access-k95f5") pod "bf1da21d-7de6-4f0e-9341-f4add2840c0a" (UID: "bf1da21d-7de6-4f0e-9341-f4add2840c0a"). InnerVolumeSpecName "kube-api-access-k95f5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:56:18.243620 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.243575    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:56:18.243620 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.243614    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:56:18.243620 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.243625    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf1da21d-7de6-4f0e-9341-f4add2840c0a-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:56:18.243620 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.243634    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k95f5\" (UniqueName: \"kubernetes.io/projected/bf1da21d-7de6-4f0e-9341-f4add2840c0a-kube-api-access-k95f5\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:56:18.243914 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.243643    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf1da21d-7de6-4f0e-9341-f4add2840c0a-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:56:18.876139 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.876109    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"
Sep 11 16:56:18.877273 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.876160    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk" event={"ID":"bf1da21d-7de6-4f0e-9341-f4add2840c0a","Type":"ContainerDied","Data":"caa24f446ded224c55a5fa8d0cd44e98d2c5d6b081493b5137b057e1428c6bc8"}
Sep 11 16:56:18.877395 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.877306    2545 scope.go:117] "RemoveContainer" containerID="ad75df29f13bfd107fd826704a03462ad5c85b51771e71e1e3164ef69aece150"
Sep 11 16:56:18.912522 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.912494    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"]
Sep 11 16:56:18.917076 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:18.917053    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ce902a/rate-test-model-ce902a-kserve-689475fb9f-zg5zk"]
Sep 11 16:56:19.250794 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:19.250760    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:56:19.939067 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:19.939034    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf1da21d-7de6-4f0e-9341-f4add2840c0a" path="/var/lib/kubelet/pods/bf1da21d-7de6-4f0e-9341-f4add2840c0a/volumes"
Sep 11 16:56:22.489001 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:22.488129    2545 patch_prober.go:28] interesting pod/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": read tcp 10.133.0.2:52796->10.133.0.93:15021: read: connection reset by peer" start-of-body=
Sep 11 16:56:22.489001 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:22.488190    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" podUID="0508d476-76f3-4dd0-b873-7392d519a751" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": read tcp 10.133.0.2:52796->10.133.0.93:15021: read: connection reset by peer"
Sep 11 16:56:22.902645 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:22.902562    2545 generic.go:356] "Generic (PLEG): container finished" podID="0508d476-76f3-4dd0-b873-7392d519a751" containerID="dc67b3bf036c525149d2e999ddc2ea8e5b44af9198c3109bdebcc4734a9bef42" exitCode=0
Sep 11 16:56:22.902789 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:22.902639    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" event={"ID":"0508d476-76f3-4dd0-b873-7392d519a751","Type":"ContainerDied","Data":"dc67b3bf036c525149d2e999ddc2ea8e5b44af9198c3109bdebcc4734a9bef42"}
Sep 11 16:56:22.903098 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:22.903079    2545 scope.go:117] "RemoveContainer" containerID="dc67b3bf036c525149d2e999ddc2ea8e5b44af9198c3109bdebcc4734a9bef42"
Sep 11 16:56:22.904161 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:22.904135    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:56:22.904224 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:22.904201    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:56:22.904261 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:22.904233    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:56:23.910433 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:23.910391    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" event={"ID":"0508d476-76f3-4dd0-b873-7392d519a751","Type":"ContainerStarted","Data":"f33c70aa543d11247b4dbbb7effb70578e1e4e2320d7be05fafbde0510d4f6e0"}
Sep 11 16:56:24.854285 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:24.854248    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:24.854503 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:24.854481    2545 patch_prober.go:28] interesting pod/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Sep 11 16:56:24.854557 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:24.854531    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" podUID="0508d476-76f3-4dd0-b873-7392d519a751" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Sep 11 16:56:25.854557 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:25.854522    2545 patch_prober.go:28] interesting pod/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Sep 11 16:56:25.855021 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:25.854578    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" podUID="0508d476-76f3-4dd0-b873-7392d519a751" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Sep 11 16:56:26.854180 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:26.854141    2545 patch_prober.go:28] interesting pod/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Sep 11 16:56:26.854383 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:26.854200    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" podUID="0508d476-76f3-4dd0-b873-7392d519a751" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Sep 11 16:56:27.854615 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:27.854581    2545 patch_prober.go:28] interesting pod/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Sep 11 16:56:27.855055 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:27.854638    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" podUID="0508d476-76f3-4dd0-b873-7392d519a751" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Sep 11 16:56:28.853842 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:28.853799    2545 patch_prober.go:28] interesting pod/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Sep 11 16:56:28.854178 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:28.853904    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" podUID="0508d476-76f3-4dd0-b873-7392d519a751" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Sep 11 16:56:29.857506 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:29.857470    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:29.858016 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:29.857821    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:29.858492 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:29.858478    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 16:56:32.855111 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.855075    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"]
Sep 11 16:56:32.855504 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.855493    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36" containerName="main"
Sep 11 16:56:32.855504 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.855505    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36" containerName="main"
Sep 11 16:56:32.855589 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.855527    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf1da21d-7de6-4f0e-9341-f4add2840c0a" containerName="main"
Sep 11 16:56:32.855589 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.855532    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf1da21d-7de6-4f0e-9341-f4add2840c0a" containerName="main"
Sep 11 16:56:32.855655 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.855616    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="62c50559-360b-451a-afe0-7ac96f1c3935" containerName="istio-proxy"
Sep 11 16:56:32.855655 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.855631    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d0a9fb5-f078-4ce3-bed7-1d2b22e44c36" containerName="main"
Sep 11 16:56:32.855655 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.855637    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf1da21d-7de6-4f0e-9341-f4add2840c0a" containerName="main"
Sep 11 16:56:32.858871 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.858845    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:32.861758 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.861739    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-eb677c-openshift-default-dockercfg-wtv4w\""
Sep 11 16:56:32.874202 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.874173    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"]
Sep 11 16:56:32.894681 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.894644    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-data\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:32.894937 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.894705    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-podinfo\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:32.894937 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.894767    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-token\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:32.894937 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.894817    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-credential-socket\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:32.894937 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.894894    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-envoy\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:32.895201 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.894937    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-workload-certs\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:32.895201 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.894989    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92wln\" (UniqueName: \"kubernetes.io/projected/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-kube-api-access-92wln\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:32.895201 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.895111    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-workload-socket\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:32.895201 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:32.895177    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istiod-ca-cert\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.004766 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.004712    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-podinfo\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.004982 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.004782    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-token\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.004982 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.004809    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-credential-socket\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.004982 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.004881    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-envoy\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.004982 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.004918    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-workload-certs\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.004982 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.004952    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92wln\" (UniqueName: \"kubernetes.io/projected/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-kube-api-access-92wln\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.005310 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.005009    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-workload-socket\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.005310 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.005058    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istiod-ca-cert\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.005310 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.005162    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-data\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.005656 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.005627    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-data\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.008181 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.008147    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-workload-certs\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.008181 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.008160    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-workload-socket\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.008719 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.008695    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-credential-socket\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.013063 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.011932    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istiod-ca-cert\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.013063 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.012459    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-podinfo\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.015439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.015414    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-envoy\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.018433 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.018408    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92wln\" (UniqueName: \"kubernetes.io/projected/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-kube-api-access-92wln\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.020382 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.020362    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-token\") pod \"e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") " pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.169687 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.169579    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:33.477707 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.477680    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"]
Sep 11 16:56:33.479590 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:56:33.479554    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69bd555b_e6f5_4d66_bf74_ddfe48d6efc3.slice/crio-2736056861f64c6ac4ffa4044aabda98acf3431255117ba6d7fb98cee1d578a4 WatchSource:0}: Error finding container 2736056861f64c6ac4ffa4044aabda98acf3431255117ba6d7fb98cee1d578a4: Status 404 returned error can't find the container with id 2736056861f64c6ac4ffa4044aabda98acf3431255117ba6d7fb98cee1d578a4
Sep 11 16:56:33.481928 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.481893    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:56:33.482042 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.481962    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:56:33.482042 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.481991    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:56:33.970339 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.970300    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" event={"ID":"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3","Type":"ContainerStarted","Data":"559cac25b00180cb38804923fc3ceab29a0b233e6d9d356d904e3dfd1bc9cc3e"}
Sep 11 16:56:33.970800 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.970362    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" event={"ID":"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3","Type":"ContainerStarted","Data":"2736056861f64c6ac4ffa4044aabda98acf3431255117ba6d7fb98cee1d578a4"}
Sep 11 16:56:33.993750 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:33.993691    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" podStartSLOduration=1.993674371 podStartE2EDuration="1.993674371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:56:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:56:33.991887515 +0000 UTC m=+1612.545057471" watchObservedRunningTime="2026-09-11 16:56:33.993674371 +0000 UTC m=+1612.546844298"
Sep 11 16:56:34.170643 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:34.170600    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:34.175980 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:34.175952    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:34.976386 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:34.976348    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:34.977429 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:34.977408    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:42.435606 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:42.435568    2545 scope.go:117] "RemoveContainer" containerID="68cc8e54acc1e46660d7e9b463073af28caa0023e16e4707200fae6e7cb789eb"
Sep 11 16:56:43.350202 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:43.350167    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:56:51.018139 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:51.018086    2545 patch_prober.go:28] interesting pod/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 16:56:51.018540 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:51.018173    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 16:56:53.083465 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:53.083369    2545 generic.go:356] "Generic (PLEG): container finished" podID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerID="559cac25b00180cb38804923fc3ceab29a0b233e6d9d356d904e3dfd1bc9cc3e" exitCode=0
Sep 11 16:56:53.083892 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:53.083449    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" event={"ID":"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3","Type":"ContainerDied","Data":"559cac25b00180cb38804923fc3ceab29a0b233e6d9d356d904e3dfd1bc9cc3e"}
Sep 11 16:56:53.084052 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:53.084035    2545 scope.go:117] "RemoveContainer" containerID="559cac25b00180cb38804923fc3ceab29a0b233e6d9d356d904e3dfd1bc9cc3e"
Sep 11 16:56:53.085343 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:53.085316    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:56:53.085419 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:53.085388    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:56:53.085471 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:53.085424    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:56:54.090930 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:54.090858    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" event={"ID":"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3","Type":"ContainerStarted","Data":"1ebf11550beb688087de1374595593bbe1084841fd5d80cc97e78edddaa9508b"}
Sep 11 16:56:54.170146 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:54.170104    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:56:54.170349 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:54.170328    2545 patch_prober.go:28] interesting pod/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Sep 11 16:56:54.170398 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:54.170375    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Sep 11 16:56:55.170282 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:55.170241    2545 patch_prober.go:28] interesting pod/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Sep 11 16:56:55.170688 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:55.170303    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Sep 11 16:56:56.170527 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:56.170482    2545 patch_prober.go:28] interesting pod/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Sep 11 16:56:56.171072 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:56.170558    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Sep 11 16:56:57.170173 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:57.170136    2545 patch_prober.go:28] interesting pod/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Sep 11 16:56:57.170341 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:57.170194    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Sep 11 16:56:58.170288 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:58.170249    2545 patch_prober.go:28] interesting pod/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Sep 11 16:56:58.170749 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:58.170308    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Sep 11 16:56:59.170358 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:59.170317    2545 patch_prober.go:28] interesting pod/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Sep 11 16:56:59.170837 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:56:59.170386    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Sep 11 16:57:00.173987 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:00.173958    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:57:00.174383 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:00.174213    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:57:00.174906 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:00.174887    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 16:57:15.208300 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.208266    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"]
Sep 11 16:57:15.212043 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.212022    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.214329 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.214308    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-519b89\"/\"openshift-service-ca.crt\""
Sep 11 16:57:15.215192 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.215164    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-519b89\"/\"ipp-route-519b89-kserve-self-signed-certs\""
Sep 11 16:57:15.215319 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.215212    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-519b89\"/\"default-dockercfg-5hvzr\""
Sep 11 16:57:15.215319 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.215191    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-519b89\"/\"kube-root-ca.crt\""
Sep 11 16:57:15.219670 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.219203    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"]
Sep 11 16:57:15.328290 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.328260    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-tmp-dir\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.328290 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.328291    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7htb6\" (UniqueName: \"kubernetes.io/projected/0ad2a79f-a7d7-407a-856f-9125122e1673-kube-api-access-7htb6\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.328487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.328335    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-home\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.328487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.328358    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ad2a79f-a7d7-407a-856f-9125122e1673-tls-certs\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.328487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.328379    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-model-cache\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.328487 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.328413    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-dshm\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.429692 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.429659    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-dshm\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.429831 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.429726    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-tmp-dir\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.429831 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.429767    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7htb6\" (UniqueName: \"kubernetes.io/projected/0ad2a79f-a7d7-407a-856f-9125122e1673-kube-api-access-7htb6\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.430007 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.429837    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-home\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.430007 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.429889    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ad2a79f-a7d7-407a-856f-9125122e1673-tls-certs\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.430007 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.429929    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-model-cache\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.430189 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.430160    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-tmp-dir\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.430255 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.430204    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-home\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.430310 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.430260    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-model-cache\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.432044 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.432026    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-dshm\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.432491 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.432470    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ad2a79f-a7d7-407a-856f-9125122e1673-tls-certs\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.439154 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.439134    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7htb6\" (UniqueName: \"kubernetes.io/projected/0ad2a79f-a7d7-407a-856f-9125122e1673-kube-api-access-7htb6\") pod \"ipp-route-519b89-kserve-6c77c679c5-c4sp6\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") " pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.523821 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.523788    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:15.799527 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:15.799501    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"]
Sep 11 16:57:15.801228 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:57:15.801201    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ad2a79f_a7d7_407a_856f_9125122e1673.slice/crio-c8b3023609b93380aa1245c333101a7ba7ee5155147a39cb8a9db466cec09d87 WatchSource:0}: Error finding container c8b3023609b93380aa1245c333101a7ba7ee5155147a39cb8a9db466cec09d87: Status 404 returned error can't find the container with id c8b3023609b93380aa1245c333101a7ba7ee5155147a39cb8a9db466cec09d87
Sep 11 16:57:16.220686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:16.220599    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6" event={"ID":"0ad2a79f-a7d7-407a-856f-9125122e1673","Type":"ContainerStarted","Data":"e52dde1317c43e58e5c673e6395b925b2f4a2c5df163cb92692e919278300dc3"}
Sep 11 16:57:16.220686 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:16.220641    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6" event={"ID":"0ad2a79f-a7d7-407a-856f-9125122e1673","Type":"ContainerStarted","Data":"c8b3023609b93380aa1245c333101a7ba7ee5155147a39cb8a9db466cec09d87"}
Sep 11 16:57:16.239026 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:16.238984    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6" podStartSLOduration=1.238969462 podStartE2EDuration="1.238969462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:57:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:57:16.237566791 +0000 UTC m=+1654.790736734" watchObservedRunningTime="2026-09-11 16:57:16.238969462 +0000 UTC m=+1654.792139389"
Sep 11 16:57:19.247463 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:19.247433    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:57:25.524046 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:25.524008    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:25.524554 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:25.524062    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:25.536758 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:25.536727    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:26.286453 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:26.286421    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:57:51.159770 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:51.159733    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:57:58.090344 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.090311    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"]
Sep 11 16:57:58.094190 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.094167    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.096590 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.096566    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-8a65bd-openshift-default-dockercfg-kpp79\""
Sep 11 16:57:58.107111 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.107083    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"]
Sep 11 16:57:58.257336 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.257295    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-workload-socket\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.257526 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.257344    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e753aa39-ef7d-4636-9599-5137cf932a78-istio-token\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.257594 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.257514    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e753aa39-ef7d-4636-9599-5137cf932a78-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.257594 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.257564    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e753aa39-ef7d-4636-9599-5137cf932a78-istio-podinfo\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.257674 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.257625    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-istio-envoy\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.257674 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.257653    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-credential-socket\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.257760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.257677    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-workload-certs\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.257760 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.257706    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khlkz\" (UniqueName: \"kubernetes.io/projected/e753aa39-ef7d-4636-9599-5137cf932a78-kube-api-access-khlkz\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.257842 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.257778    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-istio-data\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.358509 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.358432    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e753aa39-ef7d-4636-9599-5137cf932a78-istio-podinfo\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.358509 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.358468    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-istio-envoy\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.358509 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.358486    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-credential-socket\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.358509 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.358503    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-workload-certs\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.358830 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.358520    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khlkz\" (UniqueName: \"kubernetes.io/projected/e753aa39-ef7d-4636-9599-5137cf932a78-kube-api-access-khlkz\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.358830 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.358644    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-istio-data\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.358830 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.358700    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-workload-socket\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.358830 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.358735    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e753aa39-ef7d-4636-9599-5137cf932a78-istio-token\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.359152 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.358834    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e753aa39-ef7d-4636-9599-5137cf932a78-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.359152 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.359012    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-credential-socket\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.359152 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.359088    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-workload-certs\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.359482 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.359232    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-workload-socket\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.359482 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.359419    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-istio-data\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.359608 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.359591    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e753aa39-ef7d-4636-9599-5137cf932a78-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.361046 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.361022    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-istio-envoy\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.361153 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.361107    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e753aa39-ef7d-4636-9599-5137cf932a78-istio-podinfo\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.368247 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.368222    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e753aa39-ef7d-4636-9599-5137cf932a78-istio-token\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.368552 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.368531    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khlkz\" (UniqueName: \"kubernetes.io/projected/e753aa39-ef7d-4636-9599-5137cf932a78-kube-api-access-khlkz\") pod \"e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") " pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.406556 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.406531    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:57:58.688954 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.688926    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"]
Sep 11 16:57:58.691297 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:57:58.691274    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode753aa39_ef7d_4636_9599_5137cf932a78.slice/crio-5f4d6ce3dfd580c560234afab9f0924e411c6f1c268d720e78f776fe6fc337d0 WatchSource:0}: Error finding container 5f4d6ce3dfd580c560234afab9f0924e411c6f1c268d720e78f776fe6fc337d0: Status 404 returned error can't find the container with id 5f4d6ce3dfd580c560234afab9f0924e411c6f1c268d720e78f776fe6fc337d0
Sep 11 16:57:58.694066 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.694036    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:57:58.694142 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.694101    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:57:58.694142 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:58.694133    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:57:59.460918 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:59.460849    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" event={"ID":"e753aa39-ef7d-4636-9599-5137cf932a78","Type":"ContainerStarted","Data":"82cb9a6346d4a9c96d163ed7f95d42c2780fea9baf89680a6d3e6eb345ca2d33"}
Sep 11 16:57:59.461289 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:59.460949    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" event={"ID":"e753aa39-ef7d-4636-9599-5137cf932a78","Type":"ContainerStarted","Data":"5f4d6ce3dfd580c560234afab9f0924e411c6f1c268d720e78f776fe6fc337d0"}
Sep 11 16:57:59.484169 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:57:59.484118    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" podStartSLOduration=1.4841021030000001 podStartE2EDuration="1.484102103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:57:59.48373146 +0000 UTC m=+1698.036901412" watchObservedRunningTime="2026-09-11 16:57:59.484102103 +0000 UTC m=+1698.037272030"
Sep 11 16:58:00.406965 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:00.406914    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:58:00.412424 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:00.412395    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:58:00.466966 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:00.466934    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:58:00.468007 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:00.467987    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:58:11.243916 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:11.243814    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:58:21.595833 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:21.595793    2545 generic.go:356] "Generic (PLEG): container finished" podID="e753aa39-ef7d-4636-9599-5137cf932a78" containerID="82cb9a6346d4a9c96d163ed7f95d42c2780fea9baf89680a6d3e6eb345ca2d33" exitCode=0
Sep 11 16:58:21.596276 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:21.595853    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" event={"ID":"e753aa39-ef7d-4636-9599-5137cf932a78","Type":"ContainerDied","Data":"82cb9a6346d4a9c96d163ed7f95d42c2780fea9baf89680a6d3e6eb345ca2d33"}
Sep 11 16:58:21.596332 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:21.596317    2545 scope.go:117] "RemoveContainer" containerID="82cb9a6346d4a9c96d163ed7f95d42c2780fea9baf89680a6d3e6eb345ca2d33"
Sep 11 16:58:21.597675 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:21.597642    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:58:21.597740 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:21.597728    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:58:21.597781 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:21.597763    2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 16:58:22.604000 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:22.603962    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" event={"ID":"e753aa39-ef7d-4636-9599-5137cf932a78","Type":"ContainerStarted","Data":"0c3e1c438dc2294a13986d670afdf4d534839fa3bb8bbfb2b88a389afd35c0da"}
Sep 11 16:58:23.407005 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:23.406972    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:58:23.407209 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:23.407179    2545 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Sep 11 16:58:23.407272 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:23.407244    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Sep 11 16:58:24.407921 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:24.407854    2545 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Sep 11 16:58:24.408346 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:24.407947    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Sep 11 16:58:25.407584 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:25.407540    2545 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Sep 11 16:58:25.407786 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:25.407603    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Sep 11 16:58:26.407687 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:26.407644    2545 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Sep 11 16:58:26.408162 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:26.407711    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Sep 11 16:58:27.121495 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:27.121460    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf"]
Sep 11 16:58:27.125271 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:27.125252    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf"
Sep 11 16:58:27.127612 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:27.127583    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:58:27.127612 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:27.127583    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:58:27.134044 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:27.133420    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf"]
Sep 11 16:58:27.263553 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:27.263510    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b76474-0f94-4858-a35a-69d464b1e2a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf\" (UID: \"e1b76474-0f94-4858-a35a-69d464b1e2a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf"
Sep 11 16:58:27.365148 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:27.365104    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b76474-0f94-4858-a35a-69d464b1e2a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf\" (UID: \"e1b76474-0f94-4858-a35a-69d464b1e2a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf"
Sep 11 16:58:27.365763 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:27.365743    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b76474-0f94-4858-a35a-69d464b1e2a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf\" (UID: \"e1b76474-0f94-4858-a35a-69d464b1e2a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf"
Sep 11 16:58:27.407696 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:27.407598    2545 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Sep 11 16:58:27.407696 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:27.407663    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Sep 11 16:58:27.438054 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:27.438008    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf"
Sep 11 16:58:27.739026 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:27.738948    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf"]
Sep 11 16:58:27.741544 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:58:27.741509    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1b76474_0f94_4858_a35a_69d464b1e2a1.slice/crio-253b4ab24668f21845d9166276da1854ddbc98e06e81081988c34caa05fb5ee2 WatchSource:0}: Error finding container 253b4ab24668f21845d9166276da1854ddbc98e06e81081988c34caa05fb5ee2: Status 404 returned error can't find the container with id 253b4ab24668f21845d9166276da1854ddbc98e06e81081988c34caa05fb5ee2
Sep 11 16:58:28.407802 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:28.407713    2545 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Sep 11 16:58:28.407802 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:28.407778    2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Sep 11 16:58:28.642593 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:28.642551    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf" event={"ID":"e1b76474-0f94-4858-a35a-69d464b1e2a1","Type":"ContainerStarted","Data":"206b0f684ca588fbfebc46f7637912542608c8c431c4141d2aaba7006cb7079b"}
Sep 11 16:58:28.642795 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:28.642612    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf" event={"ID":"e1b76474-0f94-4858-a35a-69d464b1e2a1","Type":"ContainerStarted","Data":"253b4ab24668f21845d9166276da1854ddbc98e06e81081988c34caa05fb5ee2"}
Sep 11 16:58:28.659913 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:28.659790    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf" podStartSLOduration=1.659770606 podStartE2EDuration="1.659770606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:58:28.658465758 +0000 UTC m=+1727.211635709" watchObservedRunningTime="2026-09-11 16:58:28.659770606 +0000 UTC m=+1727.212940533"
Sep 11 16:58:29.412212 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:29.412178    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:58:29.412673 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:29.412556    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:58:29.413437 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:29.413420    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:58:29.652941 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:29.652903    2545 generic.go:356] "Generic (PLEG): container finished" podID="e1b76474-0f94-4858-a35a-69d464b1e2a1" containerID="206b0f684ca588fbfebc46f7637912542608c8c431c4141d2aaba7006cb7079b" exitCode=0
Sep 11 16:58:29.653127 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:29.652976    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf" event={"ID":"e1b76474-0f94-4858-a35a-69d464b1e2a1","Type":"ContainerDied","Data":"206b0f684ca588fbfebc46f7637912542608c8c431c4141d2aaba7006cb7079b"}
Sep 11 16:58:30.801475 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:30.801448    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf"
Sep 11 16:58:30.906688 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:30.906585    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b76474-0f94-4858-a35a-69d464b1e2a1-maas-api-service-ca\") pod \"e1b76474-0f94-4858-a35a-69d464b1e2a1\" (UID: \"e1b76474-0f94-4858-a35a-69d464b1e2a1\") "
Sep 11 16:58:30.906994 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:30.906971    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1b76474-0f94-4858-a35a-69d464b1e2a1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e1b76474-0f94-4858-a35a-69d464b1e2a1" (UID: "e1b76474-0f94-4858-a35a-69d464b1e2a1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:58:31.007754 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:31.007706    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b76474-0f94-4858-a35a-69d464b1e2a1-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:58:31.666097 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:31.666066    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf"
Sep 11 16:58:31.666280 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:31.666069    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf" event={"ID":"e1b76474-0f94-4858-a35a-69d464b1e2a1","Type":"ContainerDied","Data":"253b4ab24668f21845d9166276da1854ddbc98e06e81081988c34caa05fb5ee2"}
Sep 11 16:58:31.666280 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:31.666193    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="253b4ab24668f21845d9166276da1854ddbc98e06e81081988c34caa05fb5ee2"
Sep 11 16:58:33.823634 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:33.823602    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf"]
Sep 11 16:58:33.826459 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:33.826437    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8a65bd-h45pf"]
Sep 11 16:58:33.938081 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:33.938049    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1b76474-0f94-4858-a35a-69d464b1e2a1" path="/var/lib/kubelet/pods/e1b76474-0f94-4858-a35a-69d464b1e2a1/volumes"
Sep 11 16:58:37.749096 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:37.749061    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:58:40.913160 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:40.913130    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"]
Sep 11 16:58:40.913527 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:40.913359    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" containerName="istio-proxy" containerID="cri-o://0c3e1c438dc2294a13986d670afdf4d534839fa3bb8bbfb2b88a389afd35c0da" gracePeriod=30
Sep 11 16:58:44.415002 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:44.414964    2545 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 16:58:44.415394 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:44.415034    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 16:58:44.570933 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:44.570901    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j"]
Sep 11 16:58:44.571321 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:44.571309    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1b76474-0f94-4858-a35a-69d464b1e2a1" containerName="revoke-keys"
Sep 11 16:58:44.571367 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:44.571325    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1b76474-0f94-4858-a35a-69d464b1e2a1" containerName="revoke-keys"
Sep 11 16:58:44.571423 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:44.571414    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1b76474-0f94-4858-a35a-69d464b1e2a1" containerName="revoke-keys"
Sep 11 16:58:44.574656 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:44.574637    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j"
Sep 11 16:58:44.577008 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:44.576985    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:58:44.577373 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:44.577259    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:58:44.580971 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:44.580949    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j"]
Sep 11 16:58:44.634876 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:44.634843    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48c3c3ca-7529-4015-9989-ec9186ae85fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j\" (UID: \"48c3c3ca-7529-4015-9989-ec9186ae85fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j"
Sep 11 16:58:44.735492 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:44.735426    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48c3c3ca-7529-4015-9989-ec9186ae85fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j\" (UID: \"48c3c3ca-7529-4015-9989-ec9186ae85fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j"
Sep 11 16:58:44.736061 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:44.736042    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48c3c3ca-7529-4015-9989-ec9186ae85fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j\" (UID: \"48c3c3ca-7529-4015-9989-ec9186ae85fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j"
Sep 11 16:58:44.889580 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:44.889552    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j"
Sep 11 16:58:45.166019 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:45.165985    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j"]
Sep 11 16:58:45.170230 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:58:45.170203    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48c3c3ca_7529_4015_9989_ec9186ae85fa.slice/crio-0dcb2be100ccdd5639603b2a71f99a6b19c9742ec73fc91b389877b7647899e5 WatchSource:0}: Error finding container 0dcb2be100ccdd5639603b2a71f99a6b19c9742ec73fc91b389877b7647899e5: Status 404 returned error can't find the container with id 0dcb2be100ccdd5639603b2a71f99a6b19c9742ec73fc91b389877b7647899e5
Sep 11 16:58:45.754965 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:45.754897    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j" event={"ID":"48c3c3ca-7529-4015-9989-ec9186ae85fa","Type":"ContainerStarted","Data":"eec3f5938b1c3dbdf42bc7861c9e7d667ed0305c25080291c82002857342a33b"}
Sep 11 16:58:45.755290 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:45.754965    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j" event={"ID":"48c3c3ca-7529-4015-9989-ec9186ae85fa","Type":"ContainerStarted","Data":"0dcb2be100ccdd5639603b2a71f99a6b19c9742ec73fc91b389877b7647899e5"}
Sep 11 16:58:45.770500 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:45.770457    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j" podStartSLOduration=1.770441513 podStartE2EDuration="1.770441513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:58:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:58:45.768275961 +0000 UTC m=+1744.321445889" watchObservedRunningTime="2026-09-11 16:58:45.770441513 +0000 UTC m=+1744.323611439"
Sep 11 16:58:46.170008 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.169985    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:58:46.247583 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.247556    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-istio-envoy\") pod \"e753aa39-ef7d-4636-9599-5137cf932a78\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") "
Sep 11 16:58:46.247722 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.247609    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e753aa39-ef7d-4636-9599-5137cf932a78-istio-token\") pod \"e753aa39-ef7d-4636-9599-5137cf932a78\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") "
Sep 11 16:58:46.247722 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.247630    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e753aa39-ef7d-4636-9599-5137cf932a78-istio-podinfo\") pod \"e753aa39-ef7d-4636-9599-5137cf932a78\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") "
Sep 11 16:58:46.247722 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.247670    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e753aa39-ef7d-4636-9599-5137cf932a78-istiod-ca-cert\") pod \"e753aa39-ef7d-4636-9599-5137cf932a78\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") "
Sep 11 16:58:46.247722 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.247688    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khlkz\" (UniqueName: \"kubernetes.io/projected/e753aa39-ef7d-4636-9599-5137cf932a78-kube-api-access-khlkz\") pod \"e753aa39-ef7d-4636-9599-5137cf932a78\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") "
Sep 11 16:58:46.247959 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.247767    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-workload-socket\") pod \"e753aa39-ef7d-4636-9599-5137cf932a78\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") "
Sep 11 16:58:46.247959 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.247833    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-workload-certs\") pod \"e753aa39-ef7d-4636-9599-5137cf932a78\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") "
Sep 11 16:58:46.247959 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.247921    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-credential-socket\") pod \"e753aa39-ef7d-4636-9599-5137cf932a78\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") "
Sep 11 16:58:46.248116 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.247981    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-istio-data\") pod \"e753aa39-ef7d-4636-9599-5137cf932a78\" (UID: \"e753aa39-ef7d-4636-9599-5137cf932a78\") "
Sep 11 16:58:46.248116 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.248097    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e753aa39-ef7d-4636-9599-5137cf932a78-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e753aa39-ef7d-4636-9599-5137cf932a78" (UID: "e753aa39-ef7d-4636-9599-5137cf932a78"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:58:46.248431 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.248274    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-istio-data" (OuterVolumeSpecName: "istio-data") pod "e753aa39-ef7d-4636-9599-5137cf932a78" (UID: "e753aa39-ef7d-4636-9599-5137cf932a78"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:46.248431 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.248340    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e753aa39-ef7d-4636-9599-5137cf932a78" (UID: "e753aa39-ef7d-4636-9599-5137cf932a78"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:46.248431 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.248411    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e753aa39-ef7d-4636-9599-5137cf932a78" (UID: "e753aa39-ef7d-4636-9599-5137cf932a78"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:46.248612 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.248436    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e753aa39-ef7d-4636-9599-5137cf932a78" (UID: "e753aa39-ef7d-4636-9599-5137cf932a78"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:46.248612 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.248533    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e753aa39-ef7d-4636-9599-5137cf932a78-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:58:46.248612 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.248547    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:58:46.248612 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.248555    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:58:46.248612 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.248563    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:58:46.248612 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.248571    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:58:46.250456 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.250427    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e753aa39-ef7d-4636-9599-5137cf932a78" (UID: "e753aa39-ef7d-4636-9599-5137cf932a78"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:58:46.250456 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.250442    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e753aa39-ef7d-4636-9599-5137cf932a78-kube-api-access-khlkz" (OuterVolumeSpecName: "kube-api-access-khlkz") pod "e753aa39-ef7d-4636-9599-5137cf932a78" (UID: "e753aa39-ef7d-4636-9599-5137cf932a78"). InnerVolumeSpecName "kube-api-access-khlkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:58:46.250600 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.250473    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e753aa39-ef7d-4636-9599-5137cf932a78-istio-token" (OuterVolumeSpecName: "istio-token") pod "e753aa39-ef7d-4636-9599-5137cf932a78" (UID: "e753aa39-ef7d-4636-9599-5137cf932a78"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:58:46.250600 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.250532    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e753aa39-ef7d-4636-9599-5137cf932a78-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e753aa39-ef7d-4636-9599-5137cf932a78" (UID: "e753aa39-ef7d-4636-9599-5137cf932a78"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 16:58:46.349684 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.349614    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e753aa39-ef7d-4636-9599-5137cf932a78-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:58:46.349684 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.349639    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e753aa39-ef7d-4636-9599-5137cf932a78-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:58:46.349684 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.349652    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e753aa39-ef7d-4636-9599-5137cf932a78-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:58:46.349684 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.349663    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khlkz\" (UniqueName: \"kubernetes.io/projected/e753aa39-ef7d-4636-9599-5137cf932a78-kube-api-access-khlkz\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:58:46.762991 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.762950    2545 generic.go:356] "Generic (PLEG): container finished" podID="e753aa39-ef7d-4636-9599-5137cf932a78" containerID="0c3e1c438dc2294a13986d670afdf4d534839fa3bb8bbfb2b88a389afd35c0da" exitCode=0
Sep 11 16:58:46.763392 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.763008    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"
Sep 11 16:58:46.763392 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.763036    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" event={"ID":"e753aa39-ef7d-4636-9599-5137cf932a78","Type":"ContainerDied","Data":"0c3e1c438dc2294a13986d670afdf4d534839fa3bb8bbfb2b88a389afd35c0da"}
Sep 11 16:58:46.763392 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.763096    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr" event={"ID":"e753aa39-ef7d-4636-9599-5137cf932a78","Type":"ContainerDied","Data":"5f4d6ce3dfd580c560234afab9f0924e411c6f1c268d720e78f776fe6fc337d0"}
Sep 11 16:58:46.763392 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.763126    2545 scope.go:117] "RemoveContainer" containerID="0c3e1c438dc2294a13986d670afdf4d534839fa3bb8bbfb2b88a389afd35c0da"
Sep 11 16:58:46.764936 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.764913    2545 generic.go:356] "Generic (PLEG): container finished" podID="48c3c3ca-7529-4015-9989-ec9186ae85fa" containerID="eec3f5938b1c3dbdf42bc7861c9e7d667ed0305c25080291c82002857342a33b" exitCode=0
Sep 11 16:58:46.765050 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.765014    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j" event={"ID":"48c3c3ca-7529-4015-9989-ec9186ae85fa","Type":"ContainerDied","Data":"eec3f5938b1c3dbdf42bc7861c9e7d667ed0305c25080291c82002857342a33b"}
Sep 11 16:58:46.774082 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.774066    2545 scope.go:117] "RemoveContainer" containerID="82cb9a6346d4a9c96d163ed7f95d42c2780fea9baf89680a6d3e6eb345ca2d33"
Sep 11 16:58:46.785082 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.785062    2545 scope.go:117] "RemoveContainer" containerID="0c3e1c438dc2294a13986d670afdf4d534839fa3bb8bbfb2b88a389afd35c0da"
Sep 11 16:58:46.785409 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:58:46.785389    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c3e1c438dc2294a13986d670afdf4d534839fa3bb8bbfb2b88a389afd35c0da\": container with ID starting with 0c3e1c438dc2294a13986d670afdf4d534839fa3bb8bbfb2b88a389afd35c0da not found: ID does not exist" containerID="0c3e1c438dc2294a13986d670afdf4d534839fa3bb8bbfb2b88a389afd35c0da"
Sep 11 16:58:46.785489 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.785416    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3e1c438dc2294a13986d670afdf4d534839fa3bb8bbfb2b88a389afd35c0da"} err="failed to get container status \"0c3e1c438dc2294a13986d670afdf4d534839fa3bb8bbfb2b88a389afd35c0da\": rpc error: code = NotFound desc = could not find container \"0c3e1c438dc2294a13986d670afdf4d534839fa3bb8bbfb2b88a389afd35c0da\": container with ID starting with 0c3e1c438dc2294a13986d670afdf4d534839fa3bb8bbfb2b88a389afd35c0da not found: ID does not exist"
Sep 11 16:58:46.785489 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.785433    2545 scope.go:117] "RemoveContainer" containerID="82cb9a6346d4a9c96d163ed7f95d42c2780fea9baf89680a6d3e6eb345ca2d33"
Sep 11 16:58:46.785720 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:58:46.785704    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82cb9a6346d4a9c96d163ed7f95d42c2780fea9baf89680a6d3e6eb345ca2d33\": container with ID starting with 82cb9a6346d4a9c96d163ed7f95d42c2780fea9baf89680a6d3e6eb345ca2d33 not found: ID does not exist" containerID="82cb9a6346d4a9c96d163ed7f95d42c2780fea9baf89680a6d3e6eb345ca2d33"
Sep 11 16:58:46.785772 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.785726    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82cb9a6346d4a9c96d163ed7f95d42c2780fea9baf89680a6d3e6eb345ca2d33"} err="failed to get container status \"82cb9a6346d4a9c96d163ed7f95d42c2780fea9baf89680a6d3e6eb345ca2d33\": rpc error: code = NotFound desc = could not find container \"82cb9a6346d4a9c96d163ed7f95d42c2780fea9baf89680a6d3e6eb345ca2d33\": container with ID starting with 82cb9a6346d4a9c96d163ed7f95d42c2780fea9baf89680a6d3e6eb345ca2d33 not found: ID does not exist"
Sep 11 16:58:46.798639 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.798593    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"]
Sep 11 16:58:46.803007 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:46.802984    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8a65bd-openshift-default-886c445d4-kwjzr"]
Sep 11 16:58:47.911216 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:47.911189    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j"
Sep 11 16:58:47.942888 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:47.942840    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" path="/var/lib/kubelet/pods/e753aa39-ef7d-4636-9599-5137cf932a78/volumes"
Sep 11 16:58:47.966238 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:47.966211    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48c3c3ca-7529-4015-9989-ec9186ae85fa-maas-api-service-ca\") pod \"48c3c3ca-7529-4015-9989-ec9186ae85fa\" (UID: \"48c3c3ca-7529-4015-9989-ec9186ae85fa\") "
Sep 11 16:58:47.966535 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:47.966514    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48c3c3ca-7529-4015-9989-ec9186ae85fa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "48c3c3ca-7529-4015-9989-ec9186ae85fa" (UID: "48c3c3ca-7529-4015-9989-ec9186ae85fa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:58:47.966632 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:47.966620    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48c3c3ca-7529-4015-9989-ec9186ae85fa-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:58:48.778481 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:48.778450    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j"
Sep 11 16:58:48.778653 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:48.778496    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j" event={"ID":"48c3c3ca-7529-4015-9989-ec9186ae85fa","Type":"ContainerDied","Data":"0dcb2be100ccdd5639603b2a71f99a6b19c9742ec73fc91b389877b7647899e5"}
Sep 11 16:58:48.778653 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:48.778541    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0dcb2be100ccdd5639603b2a71f99a6b19c9742ec73fc91b389877b7647899e5"
Sep 11 16:58:57.845842 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:58:57.845802    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:59:00.538324 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:00.538274    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j"]
Sep 11 16:59:00.547441 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:00.547401    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-519b89-wwf2j"]
Sep 11 16:59:01.937485 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:01.937450    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48c3c3ca-7529-4015-9989-ec9186ae85fa" path="/var/lib/kubelet/pods/48c3c3ca-7529-4015-9989-ec9186ae85fa/volumes"
Sep 11 16:59:05.886384 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:05.886329    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"]
Sep 11 16:59:11.246605 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:11.246570    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 16:59:20.907306 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:20.907233    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6" podUID="0ad2a79f-a7d7-407a-856f-9125122e1673" containerName="main" containerID="cri-o://e52dde1317c43e58e5c673e6395b925b2f4a2c5df163cb92692e919278300dc3" gracePeriod=45
Sep 11 16:59:21.173919 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.173887    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:59:21.185573 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.185541    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-dshm\") pod \"0ad2a79f-a7d7-407a-856f-9125122e1673\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") "
Sep 11 16:59:21.185735 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.185608    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-model-cache\") pod \"0ad2a79f-a7d7-407a-856f-9125122e1673\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") "
Sep 11 16:59:21.185735 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.185667    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-tmp-dir\") pod \"0ad2a79f-a7d7-407a-856f-9125122e1673\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") "
Sep 11 16:59:21.185735 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.185700    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7htb6\" (UniqueName: \"kubernetes.io/projected/0ad2a79f-a7d7-407a-856f-9125122e1673-kube-api-access-7htb6\") pod \"0ad2a79f-a7d7-407a-856f-9125122e1673\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") "
Sep 11 16:59:21.185950 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.185739    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ad2a79f-a7d7-407a-856f-9125122e1673-tls-certs\") pod \"0ad2a79f-a7d7-407a-856f-9125122e1673\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") "
Sep 11 16:59:21.185950 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.185783    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-home\") pod \"0ad2a79f-a7d7-407a-856f-9125122e1673\" (UID: \"0ad2a79f-a7d7-407a-856f-9125122e1673\") "
Sep 11 16:59:21.185950 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.185938    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-model-cache" (OuterVolumeSpecName: "model-cache") pod "0ad2a79f-a7d7-407a-856f-9125122e1673" (UID: "0ad2a79f-a7d7-407a-856f-9125122e1673"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:59:21.186151 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.185957    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0ad2a79f-a7d7-407a-856f-9125122e1673" (UID: "0ad2a79f-a7d7-407a-856f-9125122e1673"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:59:21.186206 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.186177    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:59:21.186206 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.186192    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:59:21.186690 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.186657    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-home" (OuterVolumeSpecName: "home") pod "0ad2a79f-a7d7-407a-856f-9125122e1673" (UID: "0ad2a79f-a7d7-407a-856f-9125122e1673"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:59:21.188167 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.188143    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-dshm" (OuterVolumeSpecName: "dshm") pod "0ad2a79f-a7d7-407a-856f-9125122e1673" (UID: "0ad2a79f-a7d7-407a-856f-9125122e1673"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:59:21.188167 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.188149    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ad2a79f-a7d7-407a-856f-9125122e1673-kube-api-access-7htb6" (OuterVolumeSpecName: "kube-api-access-7htb6") pod "0ad2a79f-a7d7-407a-856f-9125122e1673" (UID: "0ad2a79f-a7d7-407a-856f-9125122e1673"). InnerVolumeSpecName "kube-api-access-7htb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:59:21.188457 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.188440    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ad2a79f-a7d7-407a-856f-9125122e1673-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0ad2a79f-a7d7-407a-856f-9125122e1673" (UID: "0ad2a79f-a7d7-407a-856f-9125122e1673"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:59:21.287644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.287599    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:59:21.287644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.287639    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7htb6\" (UniqueName: \"kubernetes.io/projected/0ad2a79f-a7d7-407a-856f-9125122e1673-kube-api-access-7htb6\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:59:21.287644 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.287650    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ad2a79f-a7d7-407a-856f-9125122e1673-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:59:21.287915 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.287663    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ad2a79f-a7d7-407a-856f-9125122e1673-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 16:59:21.968554 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.968520    2545 generic.go:356] "Generic (PLEG): container finished" podID="0ad2a79f-a7d7-407a-856f-9125122e1673" containerID="e52dde1317c43e58e5c673e6395b925b2f4a2c5df163cb92692e919278300dc3" exitCode=0
Sep 11 16:59:21.968554 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.968558    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"
Sep 11 16:59:21.969089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.968598    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6" event={"ID":"0ad2a79f-a7d7-407a-856f-9125122e1673","Type":"ContainerDied","Data":"e52dde1317c43e58e5c673e6395b925b2f4a2c5df163cb92692e919278300dc3"}
Sep 11 16:59:21.969089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.968647    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6" event={"ID":"0ad2a79f-a7d7-407a-856f-9125122e1673","Type":"ContainerDied","Data":"c8b3023609b93380aa1245c333101a7ba7ee5155147a39cb8a9db466cec09d87"}
Sep 11 16:59:21.969089 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.968667    2545 scope.go:117] "RemoveContainer" containerID="e52dde1317c43e58e5c673e6395b925b2f4a2c5df163cb92692e919278300dc3"
Sep 11 16:59:21.977785 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.977759    2545 scope.go:117] "RemoveContainer" containerID="e52dde1317c43e58e5c673e6395b925b2f4a2c5df163cb92692e919278300dc3"
Sep 11 16:59:21.978105 ip-10-0-135-58 kubenswrapper[2545]: E0911 16:59:21.978085    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e52dde1317c43e58e5c673e6395b925b2f4a2c5df163cb92692e919278300dc3\": container with ID starting with e52dde1317c43e58e5c673e6395b925b2f4a2c5df163cb92692e919278300dc3 not found: ID does not exist" containerID="e52dde1317c43e58e5c673e6395b925b2f4a2c5df163cb92692e919278300dc3"
Sep 11 16:59:21.978179 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.978118    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e52dde1317c43e58e5c673e6395b925b2f4a2c5df163cb92692e919278300dc3"} err="failed to get container status \"e52dde1317c43e58e5c673e6395b925b2f4a2c5df163cb92692e919278300dc3\": rpc error: code = NotFound desc = could not find container \"e52dde1317c43e58e5c673e6395b925b2f4a2c5df163cb92692e919278300dc3\": container with ID starting with e52dde1317c43e58e5c673e6395b925b2f4a2c5df163cb92692e919278300dc3 not found: ID does not exist"
Sep 11 16:59:21.988253 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.988225    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"]
Sep 11 16:59:21.990743 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:21.990718    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-519b89/ipp-route-519b89-kserve-6c77c679c5-c4sp6"]
Sep 11 16:59:23.939090 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:23.939050    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ad2a79f-a7d7-407a-856f-9125122e1673" path="/var/lib/kubelet/pods/0ad2a79f-a7d7-407a-856f-9125122e1673/volumes"
Sep 11 16:59:42.070788 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:42.070744    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:59:42.081006 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:42.080975    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 16:59:56.778223 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:56.778190    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"]
Sep 11 16:59:56.778681 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:56.778436    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" podUID="0508d476-76f3-4dd0-b873-7392d519a751" containerName="istio-proxy" containerID="cri-o://f33c70aa543d11247b4dbbb7effb70578e1e4e2320d7be05fafbde0510d4f6e0" gracePeriod=30
Sep 11 16:59:57.164415 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.164334    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg"]
Sep 11 16:59:57.164751 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.164738    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48c3c3ca-7529-4015-9989-ec9186ae85fa" containerName="revoke-keys"
Sep 11 16:59:57.164794 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.164753    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="48c3c3ca-7529-4015-9989-ec9186ae85fa" containerName="revoke-keys"
Sep 11 16:59:57.164794 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.164764    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" containerName="istio-proxy"
Sep 11 16:59:57.164794 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.164769    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" containerName="istio-proxy"
Sep 11 16:59:57.164794 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.164786    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ad2a79f-a7d7-407a-856f-9125122e1673" containerName="main"
Sep 11 16:59:57.164794 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.164791    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ad2a79f-a7d7-407a-856f-9125122e1673" containerName="main"
Sep 11 16:59:57.164979 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.164875    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" containerName="istio-proxy"
Sep 11 16:59:57.164979 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.164883    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ad2a79f-a7d7-407a-856f-9125122e1673" containerName="main"
Sep 11 16:59:57.164979 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.164890    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" containerName="istio-proxy"
Sep 11 16:59:57.164979 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.164895    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="48c3c3ca-7529-4015-9989-ec9186ae85fa" containerName="revoke-keys"
Sep 11 16:59:57.167994 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.167976    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg"
Sep 11 16:59:57.170180 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.170160    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 16:59:57.170317 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.170162    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 16:59:57.175640 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.175615    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg"]
Sep 11 16:59:57.248902 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.248850    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de4e58a-67b4-4adf-91c1-62502186c65e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg\" (UID: \"0de4e58a-67b4-4adf-91c1-62502186c65e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg"
Sep 11 16:59:57.349439 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.349404    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de4e58a-67b4-4adf-91c1-62502186c65e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg\" (UID: \"0de4e58a-67b4-4adf-91c1-62502186c65e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg"
Sep 11 16:59:57.350074 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.350053    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de4e58a-67b4-4adf-91c1-62502186c65e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg\" (UID: \"0de4e58a-67b4-4adf-91c1-62502186c65e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg"
Sep 11 16:59:57.481563 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.481489    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg"
Sep 11 16:59:57.754749 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:57.754719    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg"]
Sep 11 16:59:57.756913 ip-10-0-135-58 kubenswrapper[2545]: W0911 16:59:57.756855    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0de4e58a_67b4_4adf_91c1_62502186c65e.slice/crio-4961d9b47cab15fb51a302cbb9bfd7fc36ddddf7f03863216f60130385fb62fd WatchSource:0}: Error finding container 4961d9b47cab15fb51a302cbb9bfd7fc36ddddf7f03863216f60130385fb62fd: Status 404 returned error can't find the container with id 4961d9b47cab15fb51a302cbb9bfd7fc36ddddf7f03863216f60130385fb62fd
Sep 11 16:59:58.186576 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:58.186473    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg" event={"ID":"0de4e58a-67b4-4adf-91c1-62502186c65e","Type":"ContainerStarted","Data":"d3ad3718d0348755c9e689be2d1f4c3bff112faf55b65f9fa99d50edb4d04e2a"}
Sep 11 16:59:58.186576 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:58.186539    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg" event={"ID":"0de4e58a-67b4-4adf-91c1-62502186c65e","Type":"ContainerStarted","Data":"4961d9b47cab15fb51a302cbb9bfd7fc36ddddf7f03863216f60130385fb62fd"}
Sep 11 16:59:58.206945 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:58.206856    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg" podStartSLOduration=1.206835651 podStartE2EDuration="1.206835651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:59:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:59:58.205104339 +0000 UTC m=+1816.758274277" watchObservedRunningTime="2026-09-11 16:59:58.206835651 +0000 UTC m=+1816.760005578"
Sep 11 16:59:59.193846 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:59.193809    2545 generic.go:356] "Generic (PLEG): container finished" podID="0de4e58a-67b4-4adf-91c1-62502186c65e" containerID="d3ad3718d0348755c9e689be2d1f4c3bff112faf55b65f9fa99d50edb4d04e2a" exitCode=0
Sep 11 16:59:59.194240 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:59.193901    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg" event={"ID":"0de4e58a-67b4-4adf-91c1-62502186c65e","Type":"ContainerDied","Data":"d3ad3718d0348755c9e689be2d1f4c3bff112faf55b65f9fa99d50edb4d04e2a"}
Sep 11 16:59:59.860955 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:59.860918    2545 patch_prober.go:28] interesting pod/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 16:59:59.861135 ip-10-0-135-58 kubenswrapper[2545]: I0911 16:59:59.860981    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" podUID="0508d476-76f3-4dd0-b873-7392d519a751" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 17:00:00.135847 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.135758    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ce902a-29819100-75h2l"]
Sep 11 17:00:00.136217 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.136203    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" containerName="istio-proxy"
Sep 11 17:00:00.136269 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.136218    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="e753aa39-ef7d-4636-9599-5137cf932a78" containerName="istio-proxy"
Sep 11 17:00:00.139430 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.139378    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ce902a-29819100-75h2l"
Sep 11 17:00:00.142427 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.142404    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819100-grfg5"]
Sep 11 17:00:00.145683 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.145664    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819100-grfg5"
Sep 11 17:00:00.150073 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.150049    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ce902a-29819100-75h2l"
Sep 11 17:00:00.157056 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.156975    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ce902a-29819100-75h2l"]
Sep 11 17:00:00.159029 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.159001    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819100-grfg5"]
Sep 11 17:00:00.162699 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.162672    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819100-grfg5"
Sep 11 17:00:00.381053 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.381023    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg"
Sep 11 17:00:00.446122 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.446097    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ce902a-29819100-75h2l"]
Sep 11 17:00:00.449197 ip-10-0-135-58 kubenswrapper[2545]: W0911 17:00:00.449164    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c39e812_0143_409c_9f97_802f7d368bf5.slice/crio-7a379ce53af5e5361fb59d979849ce6acf48878b6763015c100b1f90bb4b4139 WatchSource:0}: Error finding container 7a379ce53af5e5361fb59d979849ce6acf48878b6763015c100b1f90bb4b4139: Status 404 returned error can't find the container with id 7a379ce53af5e5361fb59d979849ce6acf48878b6763015c100b1f90bb4b4139
Sep 11 17:00:00.466672 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.466649    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819100-grfg5"]
Sep 11 17:00:00.468307 ip-10-0-135-58 kubenswrapper[2545]: W0911 17:00:00.468266    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fcb03f3_38f4_4544_b2d1_b569df38ece9.slice/crio-de43232f71220a2e694400315318a987a855d316afbc2dc0532fe66566415499 WatchSource:0}: Error finding container de43232f71220a2e694400315318a987a855d316afbc2dc0532fe66566415499: Status 404 returned error can't find the container with id de43232f71220a2e694400315318a987a855d316afbc2dc0532fe66566415499
Sep 11 17:00:00.479615 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.479593    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de4e58a-67b4-4adf-91c1-62502186c65e-maas-api-service-ca\") pod \"0de4e58a-67b4-4adf-91c1-62502186c65e\" (UID: \"0de4e58a-67b4-4adf-91c1-62502186c65e\") "
Sep 11 17:00:00.479942 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.479920    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0de4e58a-67b4-4adf-91c1-62502186c65e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0de4e58a-67b4-4adf-91c1-62502186c65e" (UID: "0de4e58a-67b4-4adf-91c1-62502186c65e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:00:00.580426 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:00.580389    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de4e58a-67b4-4adf-91c1-62502186c65e-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:01.217506 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:01.217405    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819100-grfg5" event={"ID":"9fcb03f3-38f4-4544-b2d1-b569df38ece9","Type":"ContainerStarted","Data":"4254600d7d33565ff58126425d9d3002194efb08b6865d6cb4b4377c15b334d2"}
Sep 11 17:00:01.217506 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:01.217474    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819100-grfg5" event={"ID":"9fcb03f3-38f4-4544-b2d1-b569df38ece9","Type":"ContainerStarted","Data":"de43232f71220a2e694400315318a987a855d316afbc2dc0532fe66566415499"}
Sep 11 17:00:01.218799 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:01.218778    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg"
Sep 11 17:00:01.218924 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:01.218782    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg" event={"ID":"0de4e58a-67b4-4adf-91c1-62502186c65e","Type":"ContainerDied","Data":"4961d9b47cab15fb51a302cbb9bfd7fc36ddddf7f03863216f60130385fb62fd"}
Sep 11 17:00:01.218924 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:01.218916    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4961d9b47cab15fb51a302cbb9bfd7fc36ddddf7f03863216f60130385fb62fd"
Sep 11 17:00:01.220326 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:01.220305    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ce902a-29819100-75h2l" event={"ID":"1c39e812-0143-409c-9f97-802f7d368bf5","Type":"ContainerStarted","Data":"a8ef8e631115f27fa89ae688cb71c1320081b3ebb25be11d84e0b820694d7e74"}
Sep 11 17:00:01.220436 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:01.220348    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ce902a-29819100-75h2l" event={"ID":"1c39e812-0143-409c-9f97-802f7d368bf5","Type":"ContainerStarted","Data":"7a379ce53af5e5361fb59d979849ce6acf48878b6763015c100b1f90bb4b4139"}
Sep 11 17:00:01.236828 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:01.236772    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819100-grfg5" podStartSLOduration=1.236755874 podStartE2EDuration="1.236755874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:00:01.23630906 +0000 UTC m=+1819.789479006" watchObservedRunningTime="2026-09-11 17:00:01.236755874 +0000 UTC m=+1819.789925805"
Sep 11 17:00:01.259506 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:01.259460    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ce902a-29819100-75h2l" podStartSLOduration=1.2594471409999999 podStartE2EDuration="1.259447141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:00:01.257267674 +0000 UTC m=+1819.810437602" watchObservedRunningTime="2026-09-11 17:00:01.259447141 +0000 UTC m=+1819.812617068"
Sep 11 17:00:02.033176 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.033149    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 17:00:02.095089 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095020    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0508d476-76f3-4dd0-b873-7392d519a751-istiod-ca-cert\") pod \"0508d476-76f3-4dd0-b873-7392d519a751\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") "
Sep 11 17:00:02.095089 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095051    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-credential-socket\") pod \"0508d476-76f3-4dd0-b873-7392d519a751\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") "
Sep 11 17:00:02.095089 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095085    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-workload-certs\") pod \"0508d476-76f3-4dd0-b873-7392d519a751\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") "
Sep 11 17:00:02.095342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095115    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-istio-envoy\") pod \"0508d476-76f3-4dd0-b873-7392d519a751\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") "
Sep 11 17:00:02.095342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095151    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzh6x\" (UniqueName: \"kubernetes.io/projected/0508d476-76f3-4dd0-b873-7392d519a751-kube-api-access-wzh6x\") pod \"0508d476-76f3-4dd0-b873-7392d519a751\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") "
Sep 11 17:00:02.095342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095216    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-workload-socket\") pod \"0508d476-76f3-4dd0-b873-7392d519a751\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") "
Sep 11 17:00:02.095342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095239    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0508d476-76f3-4dd0-b873-7392d519a751-istio-podinfo\") pod \"0508d476-76f3-4dd0-b873-7392d519a751\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") "
Sep 11 17:00:02.095342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095264    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0508d476-76f3-4dd0-b873-7392d519a751-istio-token\") pod \"0508d476-76f3-4dd0-b873-7392d519a751\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") "
Sep 11 17:00:02.095342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095294    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-istio-data\") pod \"0508d476-76f3-4dd0-b873-7392d519a751\" (UID: \"0508d476-76f3-4dd0-b873-7392d519a751\") "
Sep 11 17:00:02.095665 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095332    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0508d476-76f3-4dd0-b873-7392d519a751" (UID: "0508d476-76f3-4dd0-b873-7392d519a751"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:02.095665 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095363    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0508d476-76f3-4dd0-b873-7392d519a751" (UID: "0508d476-76f3-4dd0-b873-7392d519a751"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:02.095665 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095492    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0508d476-76f3-4dd0-b873-7392d519a751-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0508d476-76f3-4dd0-b873-7392d519a751" (UID: "0508d476-76f3-4dd0-b873-7392d519a751"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:00:02.095665 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095606    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-istio-data" (OuterVolumeSpecName: "istio-data") pod "0508d476-76f3-4dd0-b873-7392d519a751" (UID: "0508d476-76f3-4dd0-b873-7392d519a751"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:02.095904 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095754    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:02.095904 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095757    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0508d476-76f3-4dd0-b873-7392d519a751" (UID: "0508d476-76f3-4dd0-b873-7392d519a751"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:02.095904 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095775    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0508d476-76f3-4dd0-b873-7392d519a751-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:02.095904 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095791    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:02.095904 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.095806    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:02.097636 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.097596    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0508d476-76f3-4dd0-b873-7392d519a751-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0508d476-76f3-4dd0-b873-7392d519a751" (UID: "0508d476-76f3-4dd0-b873-7392d519a751"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:00:02.097636 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.097611    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0508d476-76f3-4dd0-b873-7392d519a751-kube-api-access-wzh6x" (OuterVolumeSpecName: "kube-api-access-wzh6x") pod "0508d476-76f3-4dd0-b873-7392d519a751" (UID: "0508d476-76f3-4dd0-b873-7392d519a751"). InnerVolumeSpecName "kube-api-access-wzh6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:00:02.097938 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.097916    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0508d476-76f3-4dd0-b873-7392d519a751-istio-token" (OuterVolumeSpecName: "istio-token") pod "0508d476-76f3-4dd0-b873-7392d519a751" (UID: "0508d476-76f3-4dd0-b873-7392d519a751"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:00:02.097938 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.097925    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0508d476-76f3-4dd0-b873-7392d519a751" (UID: "0508d476-76f3-4dd0-b873-7392d519a751"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:02.197218 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.197192    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzh6x\" (UniqueName: \"kubernetes.io/projected/0508d476-76f3-4dd0-b873-7392d519a751-kube-api-access-wzh6x\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:02.197218 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.197216    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:02.197347 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.197225    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0508d476-76f3-4dd0-b873-7392d519a751-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:02.197347 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.197234    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0508d476-76f3-4dd0-b873-7392d519a751-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:02.197347 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.197242    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0508d476-76f3-4dd0-b873-7392d519a751-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:02.227567 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.227543    2545 generic.go:356] "Generic (PLEG): container finished" podID="0508d476-76f3-4dd0-b873-7392d519a751" containerID="f33c70aa543d11247b4dbbb7effb70578e1e4e2320d7be05fafbde0510d4f6e0" exitCode=0
Sep 11 17:00:02.227678 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.227609    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"
Sep 11 17:00:02.227678 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.227628    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" event={"ID":"0508d476-76f3-4dd0-b873-7392d519a751","Type":"ContainerDied","Data":"f33c70aa543d11247b4dbbb7effb70578e1e4e2320d7be05fafbde0510d4f6e0"}
Sep 11 17:00:02.227678 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.227673    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t" event={"ID":"0508d476-76f3-4dd0-b873-7392d519a751","Type":"ContainerDied","Data":"e366cfdf416642f8adb75a258a2e0eb256800ccf5be6e9973bf42989460deaf8"}
Sep 11 17:00:02.227847 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.227694    2545 scope.go:117] "RemoveContainer" containerID="f33c70aa543d11247b4dbbb7effb70578e1e4e2320d7be05fafbde0510d4f6e0"
Sep 11 17:00:02.229622 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.229597    2545 generic.go:356] "Generic (PLEG): container finished" podID="1c39e812-0143-409c-9f97-802f7d368bf5" containerID="a8ef8e631115f27fa89ae688cb71c1320081b3ebb25be11d84e0b820694d7e74" exitCode=0
Sep 11 17:00:02.229737 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.229689    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ce902a-29819100-75h2l" event={"ID":"1c39e812-0143-409c-9f97-802f7d368bf5","Type":"ContainerDied","Data":"a8ef8e631115f27fa89ae688cb71c1320081b3ebb25be11d84e0b820694d7e74"}
Sep 11 17:00:02.231420 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.231395    2545 generic.go:356] "Generic (PLEG): container finished" podID="9fcb03f3-38f4-4544-b2d1-b569df38ece9" containerID="4254600d7d33565ff58126425d9d3002194efb08b6865d6cb4b4377c15b334d2" exitCode=0
Sep 11 17:00:02.231541 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.231437    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819100-grfg5" event={"ID":"9fcb03f3-38f4-4544-b2d1-b569df38ece9","Type":"ContainerDied","Data":"4254600d7d33565ff58126425d9d3002194efb08b6865d6cb4b4377c15b334d2"}
Sep 11 17:00:02.238031 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.238014    2545 scope.go:117] "RemoveContainer" containerID="dc67b3bf036c525149d2e999ddc2ea8e5b44af9198c3109bdebcc4734a9bef42"
Sep 11 17:00:02.246559 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.246539    2545 scope.go:117] "RemoveContainer" containerID="f33c70aa543d11247b4dbbb7effb70578e1e4e2320d7be05fafbde0510d4f6e0"
Sep 11 17:00:02.246817 ip-10-0-135-58 kubenswrapper[2545]: E0911 17:00:02.246799    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f33c70aa543d11247b4dbbb7effb70578e1e4e2320d7be05fafbde0510d4f6e0\": container with ID starting with f33c70aa543d11247b4dbbb7effb70578e1e4e2320d7be05fafbde0510d4f6e0 not found: ID does not exist" containerID="f33c70aa543d11247b4dbbb7effb70578e1e4e2320d7be05fafbde0510d4f6e0"
Sep 11 17:00:02.246880 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.246823    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f33c70aa543d11247b4dbbb7effb70578e1e4e2320d7be05fafbde0510d4f6e0"} err="failed to get container status \"f33c70aa543d11247b4dbbb7effb70578e1e4e2320d7be05fafbde0510d4f6e0\": rpc error: code = NotFound desc = could not find container \"f33c70aa543d11247b4dbbb7effb70578e1e4e2320d7be05fafbde0510d4f6e0\": container with ID starting with f33c70aa543d11247b4dbbb7effb70578e1e4e2320d7be05fafbde0510d4f6e0 not found: ID does not exist"
Sep 11 17:00:02.246880 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.246838    2545 scope.go:117] "RemoveContainer" containerID="dc67b3bf036c525149d2e999ddc2ea8e5b44af9198c3109bdebcc4734a9bef42"
Sep 11 17:00:02.247124 ip-10-0-135-58 kubenswrapper[2545]: E0911 17:00:02.247106    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc67b3bf036c525149d2e999ddc2ea8e5b44af9198c3109bdebcc4734a9bef42\": container with ID starting with dc67b3bf036c525149d2e999ddc2ea8e5b44af9198c3109bdebcc4734a9bef42 not found: ID does not exist" containerID="dc67b3bf036c525149d2e999ddc2ea8e5b44af9198c3109bdebcc4734a9bef42"
Sep 11 17:00:02.247173 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.247129    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc67b3bf036c525149d2e999ddc2ea8e5b44af9198c3109bdebcc4734a9bef42"} err="failed to get container status \"dc67b3bf036c525149d2e999ddc2ea8e5b44af9198c3109bdebcc4734a9bef42\": rpc error: code = NotFound desc = could not find container \"dc67b3bf036c525149d2e999ddc2ea8e5b44af9198c3109bdebcc4734a9bef42\": container with ID starting with dc67b3bf036c525149d2e999ddc2ea8e5b44af9198c3109bdebcc4734a9bef42 not found: ID does not exist"
Sep 11 17:00:02.285373 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.285349    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"]
Sep 11 17:00:02.292237 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:02.292215    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-519b89-openshift-default-b55795954-kgh2t"]
Sep 11 17:00:03.387094 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:03.387070    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ce902a-29819100-75h2l"
Sep 11 17:00:03.389275 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:03.389259    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819100-grfg5"
Sep 11 17:00:03.938553 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:03.938513    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0508d476-76f3-4dd0-b873-7392d519a751" path="/var/lib/kubelet/pods/0508d476-76f3-4dd0-b873-7392d519a751/volumes"
Sep 11 17:00:04.245429 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:04.245400    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819100-grfg5"
Sep 11 17:00:04.245600 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:04.245438    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819100-grfg5" event={"ID":"9fcb03f3-38f4-4544-b2d1-b569df38ece9","Type":"ContainerDied","Data":"de43232f71220a2e694400315318a987a855d316afbc2dc0532fe66566415499"}
Sep 11 17:00:04.245600 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:04.245513    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de43232f71220a2e694400315318a987a855d316afbc2dc0532fe66566415499"
Sep 11 17:00:04.247167 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:04.247142    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ce902a-29819100-75h2l" event={"ID":"1c39e812-0143-409c-9f97-802f7d368bf5","Type":"ContainerDied","Data":"7a379ce53af5e5361fb59d979849ce6acf48878b6763015c100b1f90bb4b4139"}
Sep 11 17:00:04.247167 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:04.247165    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ce902a-29819100-75h2l"
Sep 11 17:00:04.247373 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:04.247190    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a379ce53af5e5361fb59d979849ce6acf48878b6763015c100b1f90bb4b4139"
Sep 11 17:00:08.113647 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:08.113616    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg"]
Sep 11 17:00:08.119371 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:08.119346    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eb677c-6bxxg"]
Sep 11 17:00:09.939353 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:09.939320    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0de4e58a-67b4-4adf-91c1-62502186c65e" path="/var/lib/kubelet/pods/0de4e58a-67b4-4adf-91c1-62502186c65e/volumes"
Sep 11 17:00:10.864142 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:10.864110    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:00:15.229772 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.229728    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"]
Sep 11 17:00:15.230285 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.229970    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerName="istio-proxy" containerID="cri-o://1ebf11550beb688087de1374595593bbe1084841fd5d80cc97e78edddaa9508b" gracePeriod=30
Sep 11 17:00:15.593848 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.593817    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n"]
Sep 11 17:00:15.594240 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.594227    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0508d476-76f3-4dd0-b873-7392d519a751" containerName="istio-proxy"
Sep 11 17:00:15.594285 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.594242    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="0508d476-76f3-4dd0-b873-7392d519a751" containerName="istio-proxy"
Sep 11 17:00:15.594285 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.594258    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fcb03f3-38f4-4544-b2d1-b569df38ece9" containerName="cleanup"
Sep 11 17:00:15.594285 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.594263    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fcb03f3-38f4-4544-b2d1-b569df38ece9" containerName="cleanup"
Sep 11 17:00:15.594285 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.594275    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c39e812-0143-409c-9f97-802f7d368bf5" containerName="cleanup"
Sep 11 17:00:15.594285 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.594280    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c39e812-0143-409c-9f97-802f7d368bf5" containerName="cleanup"
Sep 11 17:00:15.594445 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.594292    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0de4e58a-67b4-4adf-91c1-62502186c65e" containerName="revoke-keys"
Sep 11 17:00:15.594445 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.594297    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="0de4e58a-67b4-4adf-91c1-62502186c65e" containerName="revoke-keys"
Sep 11 17:00:15.594445 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.594357    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="0508d476-76f3-4dd0-b873-7392d519a751" containerName="istio-proxy"
Sep 11 17:00:15.594445 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.594364    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fcb03f3-38f4-4544-b2d1-b569df38ece9" containerName="cleanup"
Sep 11 17:00:15.594445 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.594374    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="0de4e58a-67b4-4adf-91c1-62502186c65e" containerName="revoke-keys"
Sep 11 17:00:15.594445 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.594381    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c39e812-0143-409c-9f97-802f7d368bf5" containerName="cleanup"
Sep 11 17:00:15.597361 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.597345    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n"
Sep 11 17:00:15.599467 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.599447    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:00:15.599541 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.599493    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 17:00:15.604901 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.604857    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n"]
Sep 11 17:00:15.719045 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.719004    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd64af78-3984-43d9-bf08-042c650ccd36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n\" (UID: \"dd64af78-3984-43d9-bf08-042c650ccd36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n"
Sep 11 17:00:15.820483 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.820436    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd64af78-3984-43d9-bf08-042c650ccd36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n\" (UID: \"dd64af78-3984-43d9-bf08-042c650ccd36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n"
Sep 11 17:00:15.821157 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.821134    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd64af78-3984-43d9-bf08-042c650ccd36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n\" (UID: \"dd64af78-3984-43d9-bf08-042c650ccd36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n"
Sep 11 17:00:15.910125 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:15.910019    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n"
Sep 11 17:00:16.203386 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:16.203356    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n"]
Sep 11 17:00:16.205181 ip-10-0-135-58 kubenswrapper[2545]: W0911 17:00:16.205142    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd64af78_3984_43d9_bf08_042c650ccd36.slice/crio-8a4e9c80960e1f14b8f7417dd634d55eb082248c7f0a9301b6c59b29a9dad8e0 WatchSource:0}: Error finding container 8a4e9c80960e1f14b8f7417dd634d55eb082248c7f0a9301b6c59b29a9dad8e0: Status 404 returned error can't find the container with id 8a4e9c80960e1f14b8f7417dd634d55eb082248c7f0a9301b6c59b29a9dad8e0
Sep 11 17:00:16.318039 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:16.318002    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n" event={"ID":"dd64af78-3984-43d9-bf08-042c650ccd36","Type":"ContainerStarted","Data":"8a4e9c80960e1f14b8f7417dd634d55eb082248c7f0a9301b6c59b29a9dad8e0"}
Sep 11 17:00:17.324970 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:17.324940    2545 generic.go:356] "Generic (PLEG): container finished" podID="dd64af78-3984-43d9-bf08-042c650ccd36" containerID="743893e9e3b00f638f40cafa01fa552025f6af3446c3e042dfd5471ae79026dd" exitCode=0
Sep 11 17:00:17.325418 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:17.325037    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n" event={"ID":"dd64af78-3984-43d9-bf08-042c650ccd36","Type":"ContainerDied","Data":"743893e9e3b00f638f40cafa01fa552025f6af3446c3e042dfd5471ae79026dd"}
Sep 11 17:00:18.471596 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:18.471565    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n"
Sep 11 17:00:18.548260 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:18.548218    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd64af78-3984-43d9-bf08-042c650ccd36-maas-api-service-ca\") pod \"dd64af78-3984-43d9-bf08-042c650ccd36\" (UID: \"dd64af78-3984-43d9-bf08-042c650ccd36\") "
Sep 11 17:00:18.548602 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:18.548579    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd64af78-3984-43d9-bf08-042c650ccd36-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dd64af78-3984-43d9-bf08-042c650ccd36" (UID: "dd64af78-3984-43d9-bf08-042c650ccd36"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:00:18.649748 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:18.649643    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd64af78-3984-43d9-bf08-042c650ccd36-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:19.338374 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:19.338344    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n"
Sep 11 17:00:19.338547 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:19.338377    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n" event={"ID":"dd64af78-3984-43d9-bf08-042c650ccd36","Type":"ContainerDied","Data":"8a4e9c80960e1f14b8f7417dd634d55eb082248c7f0a9301b6c59b29a9dad8e0"}
Sep 11 17:00:19.338547 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:19.338428    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a4e9c80960e1f14b8f7417dd634d55eb082248c7f0a9301b6c59b29a9dad8e0"
Sep 11 17:00:20.489448 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.489421    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 17:00:20.571039 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.570927    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-credential-socket\") pod \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") "
Sep 11 17:00:20.571039 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.570998    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-envoy\") pod \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") "
Sep 11 17:00:20.571039 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571031    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-podinfo\") pod \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") "
Sep 11 17:00:20.571414 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571082    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-workload-certs\") pod \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") "
Sep 11 17:00:20.571414 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571121    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-workload-socket\") pod \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") "
Sep 11 17:00:20.571414 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571201    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istiod-ca-cert\") pod \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") "
Sep 11 17:00:20.571414 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571261    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-data\") pod \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") "
Sep 11 17:00:20.571414 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571262    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" (UID: "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:20.571414 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571303    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92wln\" (UniqueName: \"kubernetes.io/projected/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-kube-api-access-92wln\") pod \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") "
Sep 11 17:00:20.571414 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571339    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-token\") pod \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\" (UID: \"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3\") "
Sep 11 17:00:20.571765 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571406    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" (UID: "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:20.571765 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571494    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" (UID: "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:20.571765 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571556    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-data" (OuterVolumeSpecName: "istio-data") pod "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" (UID: "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:20.571765 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571726    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:20.571765 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571748    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:20.571765 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571764    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:20.572126 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.571777    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:20.572126 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.572005    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" (UID: "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:00:20.573676 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.573648    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-kube-api-access-92wln" (OuterVolumeSpecName: "kube-api-access-92wln") pod "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" (UID: "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3"). InnerVolumeSpecName "kube-api-access-92wln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:00:20.573815 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.573731    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-token" (OuterVolumeSpecName: "istio-token") pod "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" (UID: "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:00:20.574089 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.574068    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" (UID: "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:20.574089 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.574073    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" (UID: "69bd555b-e6f5-4d66-bf74-ddfe48d6efc3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:00:20.672834 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.672784    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92wln\" (UniqueName: \"kubernetes.io/projected/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-kube-api-access-92wln\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:20.672834 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.672830    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:20.672834 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.672840    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:20.672834 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.672851    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:20.673134 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:20.672858    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:21.352608 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:21.352575    2545 generic.go:356] "Generic (PLEG): container finished" podID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerID="1ebf11550beb688087de1374595593bbe1084841fd5d80cc97e78edddaa9508b" exitCode=0
Sep 11 17:00:21.352826 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:21.352636    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"
Sep 11 17:00:21.352826 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:21.352661    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" event={"ID":"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3","Type":"ContainerDied","Data":"1ebf11550beb688087de1374595593bbe1084841fd5d80cc97e78edddaa9508b"}
Sep 11 17:00:21.352826 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:21.352723    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4" event={"ID":"69bd555b-e6f5-4d66-bf74-ddfe48d6efc3","Type":"ContainerDied","Data":"2736056861f64c6ac4ffa4044aabda98acf3431255117ba6d7fb98cee1d578a4"}
Sep 11 17:00:21.352826 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:21.352751    2545 scope.go:117] "RemoveContainer" containerID="1ebf11550beb688087de1374595593bbe1084841fd5d80cc97e78edddaa9508b"
Sep 11 17:00:21.368886 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:21.368841    2545 scope.go:117] "RemoveContainer" containerID="559cac25b00180cb38804923fc3ceab29a0b233e6d9d356d904e3dfd1bc9cc3e"
Sep 11 17:00:21.377897 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:21.377850    2545 scope.go:117] "RemoveContainer" containerID="1ebf11550beb688087de1374595593bbe1084841fd5d80cc97e78edddaa9508b"
Sep 11 17:00:21.378231 ip-10-0-135-58 kubenswrapper[2545]: E0911 17:00:21.378208    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ebf11550beb688087de1374595593bbe1084841fd5d80cc97e78edddaa9508b\": container with ID starting with 1ebf11550beb688087de1374595593bbe1084841fd5d80cc97e78edddaa9508b not found: ID does not exist" containerID="1ebf11550beb688087de1374595593bbe1084841fd5d80cc97e78edddaa9508b"
Sep 11 17:00:21.378326 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:21.378239    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ebf11550beb688087de1374595593bbe1084841fd5d80cc97e78edddaa9508b"} err="failed to get container status \"1ebf11550beb688087de1374595593bbe1084841fd5d80cc97e78edddaa9508b\": rpc error: code = NotFound desc = could not find container \"1ebf11550beb688087de1374595593bbe1084841fd5d80cc97e78edddaa9508b\": container with ID starting with 1ebf11550beb688087de1374595593bbe1084841fd5d80cc97e78edddaa9508b not found: ID does not exist"
Sep 11 17:00:21.378326 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:21.378258    2545 scope.go:117] "RemoveContainer" containerID="559cac25b00180cb38804923fc3ceab29a0b233e6d9d356d904e3dfd1bc9cc3e"
Sep 11 17:00:21.378570 ip-10-0-135-58 kubenswrapper[2545]: E0911 17:00:21.378556    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"559cac25b00180cb38804923fc3ceab29a0b233e6d9d356d904e3dfd1bc9cc3e\": container with ID starting with 559cac25b00180cb38804923fc3ceab29a0b233e6d9d356d904e3dfd1bc9cc3e not found: ID does not exist" containerID="559cac25b00180cb38804923fc3ceab29a0b233e6d9d356d904e3dfd1bc9cc3e"
Sep 11 17:00:21.378613 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:21.378573    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"559cac25b00180cb38804923fc3ceab29a0b233e6d9d356d904e3dfd1bc9cc3e"} err="failed to get container status \"559cac25b00180cb38804923fc3ceab29a0b233e6d9d356d904e3dfd1bc9cc3e\": rpc error: code = NotFound desc = could not find container \"559cac25b00180cb38804923fc3ceab29a0b233e6d9d356d904e3dfd1bc9cc3e\": container with ID starting with 559cac25b00180cb38804923fc3ceab29a0b233e6d9d356d904e3dfd1bc9cc3e not found: ID does not exist"
Sep 11 17:00:21.383393 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:21.383370    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"]
Sep 11 17:00:21.388331 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:21.388304    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-eb677c-openshift-default-79569bfd64-f2gl4"]
Sep 11 17:00:21.938196 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:21.938158    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" path="/var/lib/kubelet/pods/69bd555b-e6f5-4d66-bf74-ddfe48d6efc3/volumes"
Sep 11 17:00:26.538284 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:26.538246    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n"]
Sep 11 17:00:26.544141 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:26.544111    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ff7416-dx52n"]
Sep 11 17:00:27.937411 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:27.937374    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd64af78-3984-43d9-bf08-042c650ccd36" path="/var/lib/kubelet/pods/dd64af78-3984-43d9-bf08-042c650ccd36/volumes"
Sep 11 17:00:29.344070 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:29.344031    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:00:33.987592 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.987557    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb"]
Sep 11 17:00:33.991342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.988912    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0508d476-76f3-4dd0-b873-7392d519a751" containerName="istio-proxy"
Sep 11 17:00:33.991342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.988944    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="0508d476-76f3-4dd0-b873-7392d519a751" containerName="istio-proxy"
Sep 11 17:00:33.991342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.988976    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerName="istio-proxy"
Sep 11 17:00:33.991342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.988986    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerName="istio-proxy"
Sep 11 17:00:33.991342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.989018    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerName="istio-proxy"
Sep 11 17:00:33.991342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.989027    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerName="istio-proxy"
Sep 11 17:00:33.991342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.989039    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd64af78-3984-43d9-bf08-042c650ccd36" containerName="revoke-keys"
Sep 11 17:00:33.991342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.989048    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd64af78-3984-43d9-bf08-042c650ccd36" containerName="revoke-keys"
Sep 11 17:00:33.991342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.989236    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd64af78-3984-43d9-bf08-042c650ccd36" containerName="revoke-keys"
Sep 11 17:00:33.991342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.989255    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="0508d476-76f3-4dd0-b873-7392d519a751" containerName="istio-proxy"
Sep 11 17:00:33.991342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.989266    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerName="istio-proxy"
Sep 11 17:00:33.991342 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.989282    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="69bd555b-e6f5-4d66-bf74-ddfe48d6efc3" containerName="istio-proxy"
Sep 11 17:00:33.995663 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.995630    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb"
Sep 11 17:00:33.998823 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.998799    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xqmx\""
Sep 11 17:00:33.998971 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:33.998882    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:00:34.003078 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:34.003051    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6864ae97-dc54-49a5-8a34-b6284ef41216-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb\" (UID: \"6864ae97-dc54-49a5-8a34-b6284ef41216\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb"
Sep 11 17:00:34.004738 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:34.004713    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb"]
Sep 11 17:00:34.103833 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:34.103794    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6864ae97-dc54-49a5-8a34-b6284ef41216-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb\" (UID: \"6864ae97-dc54-49a5-8a34-b6284ef41216\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb"
Sep 11 17:00:34.104514 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:34.104493    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6864ae97-dc54-49a5-8a34-b6284ef41216-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb\" (UID: \"6864ae97-dc54-49a5-8a34-b6284ef41216\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb"
Sep 11 17:00:34.308935 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:34.308831    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb"
Sep 11 17:00:34.584493 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:34.584464    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb"]
Sep 11 17:00:34.586673 ip-10-0-135-58 kubenswrapper[2545]: W0911 17:00:34.586646    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6864ae97_dc54_49a5_8a34_b6284ef41216.slice/crio-8bb43d31801895d7db159f2a9706342bde6ad49144a4c6ddadb74498dfa72a7e WatchSource:0}: Error finding container 8bb43d31801895d7db159f2a9706342bde6ad49144a4c6ddadb74498dfa72a7e: Status 404 returned error can't find the container with id 8bb43d31801895d7db159f2a9706342bde6ad49144a4c6ddadb74498dfa72a7e
Sep 11 17:00:35.445396 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:35.445359    2545 generic.go:356] "Generic (PLEG): container finished" podID="6864ae97-dc54-49a5-8a34-b6284ef41216" containerID="7596720c541b937a1c5686be4bf53660482fac5283ee5a3b33eb743fc586a9a4" exitCode=0
Sep 11 17:00:35.445807 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:35.445443    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb" event={"ID":"6864ae97-dc54-49a5-8a34-b6284ef41216","Type":"ContainerDied","Data":"7596720c541b937a1c5686be4bf53660482fac5283ee5a3b33eb743fc586a9a4"}
Sep 11 17:00:35.445807 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:35.445504    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb" event={"ID":"6864ae97-dc54-49a5-8a34-b6284ef41216","Type":"ContainerStarted","Data":"8bb43d31801895d7db159f2a9706342bde6ad49144a4c6ddadb74498dfa72a7e"}
Sep 11 17:00:36.597535 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:36.597501    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb"
Sep 11 17:00:36.627068 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:36.627037    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6864ae97-dc54-49a5-8a34-b6284ef41216-maas-api-service-ca\") pod \"6864ae97-dc54-49a5-8a34-b6284ef41216\" (UID: \"6864ae97-dc54-49a5-8a34-b6284ef41216\") "
Sep 11 17:00:36.627470 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:36.627444    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6864ae97-dc54-49a5-8a34-b6284ef41216-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6864ae97-dc54-49a5-8a34-b6284ef41216" (UID: "6864ae97-dc54-49a5-8a34-b6284ef41216"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:00:36.728080 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:36.727994    2545 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6864ae97-dc54-49a5-8a34-b6284ef41216-maas-api-service-ca\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:37.458129 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:37.458096    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb"
Sep 11 17:00:37.458316 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:37.458126    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb" event={"ID":"6864ae97-dc54-49a5-8a34-b6284ef41216","Type":"ContainerDied","Data":"8bb43d31801895d7db159f2a9706342bde6ad49144a4c6ddadb74498dfa72a7e"}
Sep 11 17:00:37.458316 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:37.458174    2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8bb43d31801895d7db159f2a9706342bde6ad49144a4c6ddadb74498dfa72a7e"
Sep 11 17:00:42.500343 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:42.500299    2545 scope.go:117] "RemoveContainer" containerID="36c0c82b5fcddaf499609b3c631e791366dd693f54837db9aaa36888a009334f"
Sep 11 17:00:42.510082 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:42.510060    2545 scope.go:117] "RemoveContainer" containerID="787397c29501a57d1f1eaf16a877ef89cd34f771378ecba4f6a4d22f8f404dac"
Sep 11 17:00:42.520180 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:42.520155    2545 scope.go:117] "RemoveContainer" containerID="413344b295f83c1f768b6e466c009a6485d301c97d1759c66693a7dcbb970a47"
Sep 11 17:00:44.543288 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:44.543248    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ce902a-29819100-75h2l"]
Sep 11 17:00:44.547757 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:44.547723    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ce902a-29819100-75h2l"]
Sep 11 17:00:44.764285 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:44.764245    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb"]
Sep 11 17:00:44.769302 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:44.769272    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ce902a-rjrbb"]
Sep 11 17:00:45.938273 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:45.938239    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c39e812-0143-409c-9f97-802f7d368bf5" path="/var/lib/kubelet/pods/1c39e812-0143-409c-9f97-802f7d368bf5/volumes"
Sep 11 17:00:45.938660 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:45.938553    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6864ae97-dc54-49a5-8a34-b6284ef41216" path="/var/lib/kubelet/pods/6864ae97-dc54-49a5-8a34-b6284ef41216/volumes"
Sep 11 17:00:47.254395 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:47.254355    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:00:51.977178 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:51.977138    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"]
Sep 11 17:00:51.977674 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:51.977415    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5" podUID="497721e4-6796-4333-9908-be581343a615" containerName="istio-proxy" containerID="cri-o://f99424747695736563e865a1a749743724067899060654944e4db97cb9303bfd" gracePeriod=30
Sep 11 17:00:55.098069 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:55.098030    2545 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 17:00:55.098448 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:55.098087    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5" podUID="497721e4-6796-4333-9908-be581343a615" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 17:00:56.543880 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:56.543829    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:00:57.230389 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.230361    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 17:00:57.326178 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326095    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-workload-certs\") pod \"497721e4-6796-4333-9908-be581343a615\" (UID: \"497721e4-6796-4333-9908-be581343a615\") "
Sep 11 17:00:57.326178 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326134    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-credential-socket\") pod \"497721e4-6796-4333-9908-be581343a615\" (UID: \"497721e4-6796-4333-9908-be581343a615\") "
Sep 11 17:00:57.326178 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326177    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-istio-envoy\") pod \"497721e4-6796-4333-9908-be581343a615\" (UID: \"497721e4-6796-4333-9908-be581343a615\") "
Sep 11 17:00:57.326422 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326226    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/497721e4-6796-4333-9908-be581343a615-istiod-ca-cert\") pod \"497721e4-6796-4333-9908-be581343a615\" (UID: \"497721e4-6796-4333-9908-be581343a615\") "
Sep 11 17:00:57.326422 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326276    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/497721e4-6796-4333-9908-be581343a615-istio-token\") pod \"497721e4-6796-4333-9908-be581343a615\" (UID: \"497721e4-6796-4333-9908-be581343a615\") "
Sep 11 17:00:57.326422 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326333    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/497721e4-6796-4333-9908-be581343a615-istio-podinfo\") pod \"497721e4-6796-4333-9908-be581343a615\" (UID: \"497721e4-6796-4333-9908-be581343a615\") "
Sep 11 17:00:57.326422 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326399    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "497721e4-6796-4333-9908-be581343a615" (UID: "497721e4-6796-4333-9908-be581343a615"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:57.326602 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326447    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-istio-data\") pod \"497721e4-6796-4333-9908-be581343a615\" (UID: \"497721e4-6796-4333-9908-be581343a615\") "
Sep 11 17:00:57.326602 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326513    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-workload-socket\") pod \"497721e4-6796-4333-9908-be581343a615\" (UID: \"497721e4-6796-4333-9908-be581343a615\") "
Sep 11 17:00:57.326602 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326547    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwnl9\" (UniqueName: \"kubernetes.io/projected/497721e4-6796-4333-9908-be581343a615-kube-api-access-cwnl9\") pod \"497721e4-6796-4333-9908-be581343a615\" (UID: \"497721e4-6796-4333-9908-be581343a615\") "
Sep 11 17:00:57.326602 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326559    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "497721e4-6796-4333-9908-be581343a615" (UID: "497721e4-6796-4333-9908-be581343a615"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:57.326815 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326712    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-istio-data" (OuterVolumeSpecName: "istio-data") pod "497721e4-6796-4333-9908-be581343a615" (UID: "497721e4-6796-4333-9908-be581343a615"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:57.326815 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326776    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "497721e4-6796-4333-9908-be581343a615" (UID: "497721e4-6796-4333-9908-be581343a615"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:57.326815 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326803    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/497721e4-6796-4333-9908-be581343a615-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "497721e4-6796-4333-9908-be581343a615" (UID: "497721e4-6796-4333-9908-be581343a615"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:00:57.327002 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.326980    2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-istio-data\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:57.327053 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.327005    2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-workload-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:57.327053 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.327018    2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-workload-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:57.327053 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.327027    2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-credential-socket\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:57.327053 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.327036    2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/497721e4-6796-4333-9908-be581343a615-istiod-ca-cert\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:57.328598 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.328576    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/497721e4-6796-4333-9908-be581343a615-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "497721e4-6796-4333-9908-be581343a615" (UID: "497721e4-6796-4333-9908-be581343a615"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:00:57.329032 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.329010    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "497721e4-6796-4333-9908-be581343a615" (UID: "497721e4-6796-4333-9908-be581343a615"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:00:57.329097 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.329032    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/497721e4-6796-4333-9908-be581343a615-istio-token" (OuterVolumeSpecName: "istio-token") pod "497721e4-6796-4333-9908-be581343a615" (UID: "497721e4-6796-4333-9908-be581343a615"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:00:57.329136 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.329122    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/497721e4-6796-4333-9908-be581343a615-kube-api-access-cwnl9" (OuterVolumeSpecName: "kube-api-access-cwnl9") pod "497721e4-6796-4333-9908-be581343a615" (UID: "497721e4-6796-4333-9908-be581343a615"). InnerVolumeSpecName "kube-api-access-cwnl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:00:57.428285 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.428243    2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/497721e4-6796-4333-9908-be581343a615-istio-envoy\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:57.428285 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.428279    2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/497721e4-6796-4333-9908-be581343a615-istio-token\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:57.428285 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.428288    2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/497721e4-6796-4333-9908-be581343a615-istio-podinfo\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:57.428285 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.428297    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwnl9\" (UniqueName: \"kubernetes.io/projected/497721e4-6796-4333-9908-be581343a615-kube-api-access-cwnl9\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:00:57.570497 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.570463    2545 generic.go:356] "Generic (PLEG): container finished" podID="497721e4-6796-4333-9908-be581343a615" containerID="f99424747695736563e865a1a749743724067899060654944e4db97cb9303bfd" exitCode=0
Sep 11 17:00:57.570956 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.570514    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"
Sep 11 17:00:57.570956 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.570558    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5" event={"ID":"497721e4-6796-4333-9908-be581343a615","Type":"ContainerDied","Data":"f99424747695736563e865a1a749743724067899060654944e4db97cb9303bfd"}
Sep 11 17:00:57.570956 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.570614    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5" event={"ID":"497721e4-6796-4333-9908-be581343a615","Type":"ContainerDied","Data":"bc0bc623ea3ea5463eca3e34a897c587aa760429c1ccd5c4f2237898c87d19b9"}
Sep 11 17:00:57.570956 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.570636    2545 scope.go:117] "RemoveContainer" containerID="f99424747695736563e865a1a749743724067899060654944e4db97cb9303bfd"
Sep 11 17:00:57.581130 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.581114    2545 scope.go:117] "RemoveContainer" containerID="069f5ae77b363f162480f97d928b7e1be6b02cdec2dd4fd532381566e7512e88"
Sep 11 17:00:57.590280 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.590256    2545 scope.go:117] "RemoveContainer" containerID="f99424747695736563e865a1a749743724067899060654944e4db97cb9303bfd"
Sep 11 17:00:57.590600 ip-10-0-135-58 kubenswrapper[2545]: E0911 17:00:57.590582    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f99424747695736563e865a1a749743724067899060654944e4db97cb9303bfd\": container with ID starting with f99424747695736563e865a1a749743724067899060654944e4db97cb9303bfd not found: ID does not exist" containerID="f99424747695736563e865a1a749743724067899060654944e4db97cb9303bfd"
Sep 11 17:00:57.590646 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.590611    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f99424747695736563e865a1a749743724067899060654944e4db97cb9303bfd"} err="failed to get container status \"f99424747695736563e865a1a749743724067899060654944e4db97cb9303bfd\": rpc error: code = NotFound desc = could not find container \"f99424747695736563e865a1a749743724067899060654944e4db97cb9303bfd\": container with ID starting with f99424747695736563e865a1a749743724067899060654944e4db97cb9303bfd not found: ID does not exist"
Sep 11 17:00:57.590646 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.590631    2545 scope.go:117] "RemoveContainer" containerID="069f5ae77b363f162480f97d928b7e1be6b02cdec2dd4fd532381566e7512e88"
Sep 11 17:00:57.591040 ip-10-0-135-58 kubenswrapper[2545]: E0911 17:00:57.591018    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"069f5ae77b363f162480f97d928b7e1be6b02cdec2dd4fd532381566e7512e88\": container with ID starting with 069f5ae77b363f162480f97d928b7e1be6b02cdec2dd4fd532381566e7512e88 not found: ID does not exist" containerID="069f5ae77b363f162480f97d928b7e1be6b02cdec2dd4fd532381566e7512e88"
Sep 11 17:00:57.591118 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.591047    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"069f5ae77b363f162480f97d928b7e1be6b02cdec2dd4fd532381566e7512e88"} err="failed to get container status \"069f5ae77b363f162480f97d928b7e1be6b02cdec2dd4fd532381566e7512e88\": rpc error: code = NotFound desc = could not find container \"069f5ae77b363f162480f97d928b7e1be6b02cdec2dd4fd532381566e7512e88\": container with ID starting with 069f5ae77b363f162480f97d928b7e1be6b02cdec2dd4fd532381566e7512e88 not found: ID does not exist"
Sep 11 17:00:57.595582 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.595555    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"]
Sep 11 17:00:57.599836 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.599811    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ce902a-openshift-default-66d69c9d-kwvt5"]
Sep 11 17:00:57.938247 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:00:57.938162    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="497721e4-6796-4333-9908-be581343a615" path="/var/lib/kubelet/pods/497721e4-6796-4333-9908-be581343a615/volumes"
Sep 11 17:01:42.559395 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:01:42.559364    2545 scope.go:117] "RemoveContainer" containerID="2129f298244a90bcbf4f0b7e2c93a1eb7a4f018668f2d23d51883541d5f7e0d4"
Sep 11 17:02:17.639854 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:02:17.639815    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:02:45.745156 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:02:45.745078    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:02:53.639545 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:02:53.639509    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:03:01.746250 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:01.746217    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:03:21.751591 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:21.751552    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:03:30.144329 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:30.144294    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:03:33.311546 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:33.311511    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"]
Sep 11 17:03:33.312062 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:33.311764    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r" podUID="589ab627-9b5f-4456-901b-7233b0e911c6" containerName="manager" containerID="cri-o://bda93c206f8227f00d02ed26d14befed0b6105ea9d1652a46b2eb8d0c391fd25" gracePeriod=10
Sep 11 17:03:35.112841 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.112791    2545 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r" podUID="589ab627-9b5f-4456-901b-7233b0e911c6" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.34:8081/readyz\": dial tcp 10.133.0.34:8081: connect: connection refused"
Sep 11 17:03:35.274697 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.274672    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"
Sep 11 17:03:35.384010 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.383899    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t62x\" (UniqueName: \"kubernetes.io/projected/589ab627-9b5f-4456-901b-7233b0e911c6-kube-api-access-6t62x\") pod \"589ab627-9b5f-4456-901b-7233b0e911c6\" (UID: \"589ab627-9b5f-4456-901b-7233b0e911c6\") "
Sep 11 17:03:35.384172 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.384019    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/589ab627-9b5f-4456-901b-7233b0e911c6-extensions-socket-volume\") pod \"589ab627-9b5f-4456-901b-7233b0e911c6\" (UID: \"589ab627-9b5f-4456-901b-7233b0e911c6\") "
Sep 11 17:03:35.384421 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.384400    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589ab627-9b5f-4456-901b-7233b0e911c6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "589ab627-9b5f-4456-901b-7233b0e911c6" (UID: "589ab627-9b5f-4456-901b-7233b0e911c6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:03:35.386352 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.386322    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/589ab627-9b5f-4456-901b-7233b0e911c6-kube-api-access-6t62x" (OuterVolumeSpecName: "kube-api-access-6t62x") pod "589ab627-9b5f-4456-901b-7233b0e911c6" (UID: "589ab627-9b5f-4456-901b-7233b0e911c6"). InnerVolumeSpecName "kube-api-access-6t62x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:03:35.472091 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.472055    2545 generic.go:356] "Generic (PLEG): container finished" podID="589ab627-9b5f-4456-901b-7233b0e911c6" containerID="bda93c206f8227f00d02ed26d14befed0b6105ea9d1652a46b2eb8d0c391fd25" exitCode=0
Sep 11 17:03:35.472262 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.472104    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"
Sep 11 17:03:35.472262 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.472130    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r" event={"ID":"589ab627-9b5f-4456-901b-7233b0e911c6","Type":"ContainerDied","Data":"bda93c206f8227f00d02ed26d14befed0b6105ea9d1652a46b2eb8d0c391fd25"}
Sep 11 17:03:35.472262 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.472218    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r" event={"ID":"589ab627-9b5f-4456-901b-7233b0e911c6","Type":"ContainerDied","Data":"02ecbb73ea5c094221cff68d3ee47def6b6a23eb6f305af3df34bec0f76d6708"}
Sep 11 17:03:35.472262 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.472247    2545 scope.go:117] "RemoveContainer" containerID="bda93c206f8227f00d02ed26d14befed0b6105ea9d1652a46b2eb8d0c391fd25"
Sep 11 17:03:35.482338 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.482321    2545 scope.go:117] "RemoveContainer" containerID="bda93c206f8227f00d02ed26d14befed0b6105ea9d1652a46b2eb8d0c391fd25"
Sep 11 17:03:35.482617 ip-10-0-135-58 kubenswrapper[2545]: E0911 17:03:35.482596    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bda93c206f8227f00d02ed26d14befed0b6105ea9d1652a46b2eb8d0c391fd25\": container with ID starting with bda93c206f8227f00d02ed26d14befed0b6105ea9d1652a46b2eb8d0c391fd25 not found: ID does not exist" containerID="bda93c206f8227f00d02ed26d14befed0b6105ea9d1652a46b2eb8d0c391fd25"
Sep 11 17:03:35.482670 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.482628    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bda93c206f8227f00d02ed26d14befed0b6105ea9d1652a46b2eb8d0c391fd25"} err="failed to get container status \"bda93c206f8227f00d02ed26d14befed0b6105ea9d1652a46b2eb8d0c391fd25\": rpc error: code = NotFound desc = could not find container \"bda93c206f8227f00d02ed26d14befed0b6105ea9d1652a46b2eb8d0c391fd25\": container with ID starting with bda93c206f8227f00d02ed26d14befed0b6105ea9d1652a46b2eb8d0c391fd25 not found: ID does not exist"
Sep 11 17:03:35.485046 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.485026    2545 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/589ab627-9b5f-4456-901b-7233b0e911c6-extensions-socket-volume\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:03:35.485046 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.485047    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t62x\" (UniqueName: \"kubernetes.io/projected/589ab627-9b5f-4456-901b-7233b0e911c6-kube-api-access-6t62x\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:03:35.493972 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.493950    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"]
Sep 11 17:03:35.498065 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.498044    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-s965r"]
Sep 11 17:03:35.937713 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:03:35.937669    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="589ab627-9b5f-4456-901b-7233b0e911c6" path="/var/lib/kubelet/pods/589ab627-9b5f-4456-901b-7233b0e911c6/volumes"
Sep 11 17:04:34.026392 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.026356    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn"]
Sep 11 17:04:34.026829 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.026738    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6864ae97-dc54-49a5-8a34-b6284ef41216" containerName="revoke-keys"
Sep 11 17:04:34.026829 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.026749    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="6864ae97-dc54-49a5-8a34-b6284ef41216" containerName="revoke-keys"
Sep 11 17:04:34.026829 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.026771    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="589ab627-9b5f-4456-901b-7233b0e911c6" containerName="manager"
Sep 11 17:04:34.026829 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.026777    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="589ab627-9b5f-4456-901b-7233b0e911c6" containerName="manager"
Sep 11 17:04:34.026829 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.026784    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="497721e4-6796-4333-9908-be581343a615" containerName="istio-proxy"
Sep 11 17:04:34.026829 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.026789    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="497721e4-6796-4333-9908-be581343a615" containerName="istio-proxy"
Sep 11 17:04:34.026829 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.026796    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="497721e4-6796-4333-9908-be581343a615" containerName="istio-proxy"
Sep 11 17:04:34.026829 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.026801    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="497721e4-6796-4333-9908-be581343a615" containerName="istio-proxy"
Sep 11 17:04:34.027117 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.026880    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="497721e4-6796-4333-9908-be581343a615" containerName="istio-proxy"
Sep 11 17:04:34.027117 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.026890    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="6864ae97-dc54-49a5-8a34-b6284ef41216" containerName="revoke-keys"
Sep 11 17:04:34.027117 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.026898    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="589ab627-9b5f-4456-901b-7233b0e911c6" containerName="manager"
Sep 11 17:04:34.027117 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.026905    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="497721e4-6796-4333-9908-be581343a615" containerName="istio-proxy"
Sep 11 17:04:34.030104 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.030085    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn"
Sep 11 17:04:34.032765 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.032742    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hzxn5\""
Sep 11 17:04:34.050720 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.050695    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn"]
Sep 11 17:04:34.110903 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.110878    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mghmd\" (UniqueName: \"kubernetes.io/projected/049a553b-8d4a-4376-8b00-9950b7db7b61-kube-api-access-mghmd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q75xn\" (UID: \"049a553b-8d4a-4376-8b00-9950b7db7b61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn"
Sep 11 17:04:34.111025 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.110996    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/049a553b-8d4a-4376-8b00-9950b7db7b61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q75xn\" (UID: \"049a553b-8d4a-4376-8b00-9950b7db7b61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn"
Sep 11 17:04:34.212211 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.212180    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/049a553b-8d4a-4376-8b00-9950b7db7b61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q75xn\" (UID: \"049a553b-8d4a-4376-8b00-9950b7db7b61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn"
Sep 11 17:04:34.212305 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.212234    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mghmd\" (UniqueName: \"kubernetes.io/projected/049a553b-8d4a-4376-8b00-9950b7db7b61-kube-api-access-mghmd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q75xn\" (UID: \"049a553b-8d4a-4376-8b00-9950b7db7b61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn"
Sep 11 17:04:34.212548 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.212527    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/049a553b-8d4a-4376-8b00-9950b7db7b61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q75xn\" (UID: \"049a553b-8d4a-4376-8b00-9950b7db7b61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn"
Sep 11 17:04:34.226211 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.226183    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mghmd\" (UniqueName: \"kubernetes.io/projected/049a553b-8d4a-4376-8b00-9950b7db7b61-kube-api-access-mghmd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q75xn\" (UID: \"049a553b-8d4a-4376-8b00-9950b7db7b61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn"
Sep 11 17:04:34.341139 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.341075    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn"
Sep 11 17:04:34.645512 ip-10-0-135-58 kubenswrapper[2545]: W0911 17:04:34.645485    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod049a553b_8d4a_4376_8b00_9950b7db7b61.slice/crio-40643064c977cf9d3ec9573d8d18bb600996e82a07c689306858ad7f69f83955 WatchSource:0}: Error finding container 40643064c977cf9d3ec9573d8d18bb600996e82a07c689306858ad7f69f83955: Status 404 returned error can't find the container with id 40643064c977cf9d3ec9573d8d18bb600996e82a07c689306858ad7f69f83955
Sep 11 17:04:34.649290 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.649270    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn"]
Sep 11 17:04:34.796066 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.796038    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn" event={"ID":"049a553b-8d4a-4376-8b00-9950b7db7b61","Type":"ContainerStarted","Data":"f5013294cbf0545f85e914029894f096d0262bb55f853946d011a346d1bf90c2"}
Sep 11 17:04:34.796236 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.796089    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn" event={"ID":"049a553b-8d4a-4376-8b00-9950b7db7b61","Type":"ContainerStarted","Data":"40643064c977cf9d3ec9573d8d18bb600996e82a07c689306858ad7f69f83955"}
Sep 11 17:04:34.796236 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.796165    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn"
Sep 11 17:04:34.818962 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:34.818911    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn" podStartSLOduration=0.818894163 podStartE2EDuration="818.894163ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:04:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:04:34.817995727 +0000 UTC m=+2093.371165654" watchObservedRunningTime="2026-09-11 17:04:34.818894163 +0000 UTC m=+2093.372064092"
Sep 11 17:04:42.106324 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:42.106292    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 17:04:42.120204 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:42.120182    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 17:04:42.604917 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:42.604883    2545 scope.go:117] "RemoveContainer" containerID="206b0f684ca588fbfebc46f7637912542608c8c431c4141d2aaba7006cb7079b"
Sep 11 17:04:45.803578 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:45.803544    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q75xn"
Sep 11 17:04:56.044879 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:04:56.044837    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:05:04.139159 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:04.139123    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:05:10.146075 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:10.146035    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:05:15.100894 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.100832    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"]
Sep 11 17:05:15.104470 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.104450    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.109904 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.106964    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-d07bc85f-kserve-self-signed-certs\""
Sep 11 17:05:15.123127 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.123103    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"]
Sep 11 17:05:15.258154 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.258118    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9183c69-b7a9-4140-b7e6-93a51fe1c900-tls-certs\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.258297 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.258204    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-model-cache\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.258297 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.258269    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-home\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.258388 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.258306    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-dshm\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.258388 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.258331    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-tmp-dir\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.258388 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.258355    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbdwg\" (UniqueName: \"kubernetes.io/projected/e9183c69-b7a9-4140-b7e6-93a51fe1c900-kube-api-access-dbdwg\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.359616 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.359538    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9183c69-b7a9-4140-b7e6-93a51fe1c900-tls-certs\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.359616 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.359589    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-model-cache\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.359796 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.359637    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-home\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.359796 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.359661    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-dshm\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.359796 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.359683    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-tmp-dir\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.359796 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.359707    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbdwg\" (UniqueName: \"kubernetes.io/projected/e9183c69-b7a9-4140-b7e6-93a51fe1c900-kube-api-access-dbdwg\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.360101 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.360081    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-model-cache\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.360216 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.360189    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-tmp-dir\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.360216 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.360206    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-home\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.362159 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.362140    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-dshm\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.362443 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.362421    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9183c69-b7a9-4140-b7e6-93a51fe1c900-tls-certs\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.370749 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.370727    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbdwg\" (UniqueName: \"kubernetes.io/projected/e9183c69-b7a9-4140-b7e6-93a51fe1c900-kube-api-access-dbdwg\") pod \"e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") " pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.423381 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.423356    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:15.699341 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:15.699263    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"]
Sep 11 17:05:15.712009 ip-10-0-135-58 kubenswrapper[2545]: W0911 17:05:15.711973    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9183c69_b7a9_4140_b7e6_93a51fe1c900.slice/crio-d1059ae5f2a0244e7ac0a6259bf288b4f3f79eaa567a813ac48aa311eae7be81 WatchSource:0}: Error finding container d1059ae5f2a0244e7ac0a6259bf288b4f3f79eaa567a813ac48aa311eae7be81: Status 404 returned error can't find the container with id d1059ae5f2a0244e7ac0a6259bf288b4f3f79eaa567a813ac48aa311eae7be81
Sep 11 17:05:16.025376 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:16.025340    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5" event={"ID":"e9183c69-b7a9-4140-b7e6-93a51fe1c900","Type":"ContainerStarted","Data":"1f391574b753cf9c062e02b131b141a862b4f8aaa2cf058b98a6162c2ef29053"}
Sep 11 17:05:16.025534 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:16.025389    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5" event={"ID":"e9183c69-b7a9-4140-b7e6-93a51fe1c900","Type":"ContainerStarted","Data":"d1059ae5f2a0244e7ac0a6259bf288b4f3f79eaa567a813ac48aa311eae7be81"}
Sep 11 17:05:16.045100 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:16.045056    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5" podStartSLOduration=1.045040081 podStartE2EDuration="1.045040081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:05:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:05:16.04360239 +0000 UTC m=+2134.596772317" watchObservedRunningTime="2026-09-11 17:05:16.045040081 +0000 UTC m=+2134.598210036"
Sep 11 17:05:17.849466 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:17.849432    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:05:25.159548 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:25.159517    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:05:25.423906 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:25.423791    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:25.423906 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:25.423833    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:25.436230 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:25.436200    2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:26.104573 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:26.104541    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:05:42.629850 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:42.629784    2545 scope.go:117] "RemoveContainer" containerID="eec3f5938b1c3dbdf42bc7861c9e7d667ed0305c25080291c82002857342a33b"
Sep 11 17:05:45.937812 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:45.937780    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"]
Sep 11 17:05:48.940732 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:48.940697    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:05:56.847067 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:05:56.847027    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:06:00.956503 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:00.956449    2545 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5" podUID="e9183c69-b7a9-4140-b7e6-93a51fe1c900" containerName="main" containerID="cri-o://1f391574b753cf9c062e02b131b141a862b4f8aaa2cf058b98a6162c2ef29053" gracePeriod=45
Sep 11 17:06:01.198541 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.198516    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:06:01.267424 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.267392    2545 generic.go:356] "Generic (PLEG): container finished" podID="e9183c69-b7a9-4140-b7e6-93a51fe1c900" containerID="1f391574b753cf9c062e02b131b141a862b4f8aaa2cf058b98a6162c2ef29053" exitCode=0
Sep 11 17:06:01.267571 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.267427    2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"
Sep 11 17:06:01.267571 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.267468    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5" event={"ID":"e9183c69-b7a9-4140-b7e6-93a51fe1c900","Type":"ContainerDied","Data":"1f391574b753cf9c062e02b131b141a862b4f8aaa2cf058b98a6162c2ef29053"}
Sep 11 17:06:01.267571 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.267511    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5" event={"ID":"e9183c69-b7a9-4140-b7e6-93a51fe1c900","Type":"ContainerDied","Data":"d1059ae5f2a0244e7ac0a6259bf288b4f3f79eaa567a813ac48aa311eae7be81"}
Sep 11 17:06:01.267571 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.267529    2545 scope.go:117] "RemoveContainer" containerID="1f391574b753cf9c062e02b131b141a862b4f8aaa2cf058b98a6162c2ef29053"
Sep 11 17:06:01.276818 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.276801    2545 scope.go:117] "RemoveContainer" containerID="1f391574b753cf9c062e02b131b141a862b4f8aaa2cf058b98a6162c2ef29053"
Sep 11 17:06:01.277134 ip-10-0-135-58 kubenswrapper[2545]: E0911 17:06:01.277116    2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f391574b753cf9c062e02b131b141a862b4f8aaa2cf058b98a6162c2ef29053\": container with ID starting with 1f391574b753cf9c062e02b131b141a862b4f8aaa2cf058b98a6162c2ef29053 not found: ID does not exist" containerID="1f391574b753cf9c062e02b131b141a862b4f8aaa2cf058b98a6162c2ef29053"
Sep 11 17:06:01.277192 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.277143    2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f391574b753cf9c062e02b131b141a862b4f8aaa2cf058b98a6162c2ef29053"} err="failed to get container status \"1f391574b753cf9c062e02b131b141a862b4f8aaa2cf058b98a6162c2ef29053\": rpc error: code = NotFound desc = could not find container \"1f391574b753cf9c062e02b131b141a862b4f8aaa2cf058b98a6162c2ef29053\": container with ID starting with 1f391574b753cf9c062e02b131b141a862b4f8aaa2cf058b98a6162c2ef29053 not found: ID does not exist"
Sep 11 17:06:01.351774 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.351737    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-model-cache\") pod \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") "
Sep 11 17:06:01.351971 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.351800    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-dshm\") pod \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") "
Sep 11 17:06:01.351971 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.351893    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbdwg\" (UniqueName: \"kubernetes.io/projected/e9183c69-b7a9-4140-b7e6-93a51fe1c900-kube-api-access-dbdwg\") pod \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") "
Sep 11 17:06:01.351971 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.351950    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9183c69-b7a9-4140-b7e6-93a51fe1c900-tls-certs\") pod \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") "
Sep 11 17:06:01.352152 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.352073    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-model-cache" (OuterVolumeSpecName: "model-cache") pod "e9183c69-b7a9-4140-b7e6-93a51fe1c900" (UID: "e9183c69-b7a9-4140-b7e6-93a51fe1c900"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:06:01.352152 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.352091    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-tmp-dir\") pod \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") "
Sep 11 17:06:01.352261 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.352178    2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-home\") pod \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\" (UID: \"e9183c69-b7a9-4140-b7e6-93a51fe1c900\") "
Sep 11 17:06:01.352317 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.352298    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e9183c69-b7a9-4140-b7e6-93a51fe1c900" (UID: "e9183c69-b7a9-4140-b7e6-93a51fe1c900"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:06:01.352457 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.352423    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-home" (OuterVolumeSpecName: "home") pod "e9183c69-b7a9-4140-b7e6-93a51fe1c900" (UID: "e9183c69-b7a9-4140-b7e6-93a51fe1c900"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:06:01.352751 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.352709    2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-model-cache\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:06:01.352751 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.352729    2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-tmp-dir\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:06:01.352751 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.352741    2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-home\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:06:01.354255 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.354231    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-dshm" (OuterVolumeSpecName: "dshm") pod "e9183c69-b7a9-4140-b7e6-93a51fe1c900" (UID: "e9183c69-b7a9-4140-b7e6-93a51fe1c900"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:06:01.354341 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.354270    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9183c69-b7a9-4140-b7e6-93a51fe1c900-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e9183c69-b7a9-4140-b7e6-93a51fe1c900" (UID: "e9183c69-b7a9-4140-b7e6-93a51fe1c900"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:06:01.354341 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.354269    2545 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9183c69-b7a9-4140-b7e6-93a51fe1c900-kube-api-access-dbdwg" (OuterVolumeSpecName: "kube-api-access-dbdwg") pod "e9183c69-b7a9-4140-b7e6-93a51fe1c900" (UID: "e9183c69-b7a9-4140-b7e6-93a51fe1c900"). InnerVolumeSpecName "kube-api-access-dbdwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:06:01.453828 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.453800    2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbdwg\" (UniqueName: \"kubernetes.io/projected/e9183c69-b7a9-4140-b7e6-93a51fe1c900-kube-api-access-dbdwg\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:06:01.453828 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.453822    2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9183c69-b7a9-4140-b7e6-93a51fe1c900-tls-certs\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:06:01.453828 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.453833    2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9183c69-b7a9-4140-b7e6-93a51fe1c900-dshm\") on node \"ip-10-0-135-58.ec2.internal\" DevicePath \"\""
Sep 11 17:06:01.591815 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.591791    2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"]
Sep 11 17:06:01.596505 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.596480    2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-d07bc85f-kserve-6957fdbdbc-drhv5"]
Sep 11 17:06:01.937845 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:01.937769    2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9183c69-b7a9-4140-b7e6-93a51fe1c900" path="/var/lib/kubelet/pods/e9183c69-b7a9-4140-b7e6-93a51fe1c900/volumes"
Sep 11 17:06:06.043710 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:06.043674    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:06:14.068506 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:14.068467    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:06:21.048628 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:21.048595    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:06:29.045968 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:29.045933    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:06:34.236363 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:34.236324    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qgz2d"]
Sep 11 17:06:42.566632 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:42.566592    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bhszh_47711732-e59c-4fe5-8650-f8f97b126a14/manager/0.log"
Sep 11 17:06:42.649197 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:42.649163    2545 scope.go:117] "RemoveContainer" containerID="d3ad3718d0348755c9e689be2d1f4c3bff112faf55b65f9fa99d50edb4d04e2a"
Sep 11 17:06:42.657633 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:42.657609    2545 scope.go:117] "RemoveContainer" containerID="7596720c541b937a1c5686be4bf53660482fac5283ee5a3b33eb743fc586a9a4"
Sep 11 17:06:42.665606 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:42.665589    2545 scope.go:117] "RemoveContainer" containerID="743893e9e3b00f638f40cafa01fa552025f6af3446c3e042dfd5471ae79026dd"
Sep 11 17:06:42.673018 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:42.672995    2545 scope.go:117] "RemoveContainer" containerID="a8ef8e631115f27fa89ae688cb71c1320081b3ebb25be11d84e0b820694d7e74"
Sep 11 17:06:42.938776 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:42.938699    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-j98b2_10039705-5220-481d-8445-a97e9eb6285d/server/0.log"
Sep 11 17:06:43.063515 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:43.063483    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-trq6d_381b5264-ef15-4c99-9e2c-924f70fed40f/manager/0.log"
Sep 11 17:06:43.438022 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:43.437993    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665d69bffd-dpbh5_f2a8594c-3871-41a0-b96f-73b0c0175bfa/copy-manifests/0.log"
Sep 11 17:06:43.445963 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:43.445934    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665d69bffd-dpbh5_f2a8594c-3871-41a0-b96f-73b0c0175bfa/manager/0.log"
Sep 11 17:06:44.029129 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:44.029101    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29819085-pdbhh_79fd3e82-3bde-4971-a3c4-2af91531d3d4/cleanup/0.log"
Sep 11 17:06:44.140498 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:44.140474    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29819100-grfg5_9fcb03f3-38f4-4544-b2d1-b569df38ece9/cleanup/0.log"
Sep 11 17:06:45.488633 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:45.488599    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-q75xn_049a553b-8d4a-4376-8b00-9950b7db7b61/manager/0.log"
Sep 11 17:06:45.603411 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:45.603379    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-qgz2d_3df23723-01c6-4388-9708-8179b50c71fb/limitador/0.log"
Sep 11 17:06:46.081479 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:46.081446    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p_92d8390b-90e0-4d31-ac76-b6496a647f85/istio-proxy/0.log"
Sep 11 17:06:47.405780 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:47.405743    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-k7mcb_9ff49a99-a74b-4a40-95de-e3e53b32df77/main/0.log"
Sep 11 17:06:47.628165 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:47.628130    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-wb4md_c7aba878-c7c8-481f-bc5c-074d731aad3d/main/0.log"
Sep 11 17:06:47.975755 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:47.975704    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-mp7m6_25680cbe-a8cb-4083-b794-ebcd1fe40e83/main/0.log"
Sep 11 17:06:53.565750 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.565718    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-df522/must-gather-qk869"]
Sep 11 17:06:53.566298 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.566181    2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9183c69-b7a9-4140-b7e6-93a51fe1c900" containerName="main"
Sep 11 17:06:53.566298 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.566196    2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9183c69-b7a9-4140-b7e6-93a51fe1c900" containerName="main"
Sep 11 17:06:53.566298 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.566252    2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9183c69-b7a9-4140-b7e6-93a51fe1c900" containerName="main"
Sep 11 17:06:53.569298 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.569282    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-df522/must-gather-qk869"
Sep 11 17:06:53.571425 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.571402    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-df522\"/\"kube-root-ca.crt\""
Sep 11 17:06:53.571558 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.571402    2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-df522\"/\"default-dockercfg-m8sb2\""
Sep 11 17:06:53.571761 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.571744    2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-df522\"/\"openshift-service-ca.crt\""
Sep 11 17:06:53.583412 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.583387    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-df522/must-gather-qk869"]
Sep 11 17:06:53.631100 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.631058    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms7jw\" (UniqueName: \"kubernetes.io/projected/cc1f65e4-2649-4f0c-828d-2e6699fbd207-kube-api-access-ms7jw\") pod \"must-gather-qk869\" (UID: \"cc1f65e4-2649-4f0c-828d-2e6699fbd207\") " pod="openshift-must-gather-df522/must-gather-qk869"
Sep 11 17:06:53.631273 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.631141    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cc1f65e4-2649-4f0c-828d-2e6699fbd207-must-gather-output\") pod \"must-gather-qk869\" (UID: \"cc1f65e4-2649-4f0c-828d-2e6699fbd207\") " pod="openshift-must-gather-df522/must-gather-qk869"
Sep 11 17:06:53.731708 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.731667    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms7jw\" (UniqueName: \"kubernetes.io/projected/cc1f65e4-2649-4f0c-828d-2e6699fbd207-kube-api-access-ms7jw\") pod \"must-gather-qk869\" (UID: \"cc1f65e4-2649-4f0c-828d-2e6699fbd207\") " pod="openshift-must-gather-df522/must-gather-qk869"
Sep 11 17:06:53.731932 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.731765    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cc1f65e4-2649-4f0c-828d-2e6699fbd207-must-gather-output\") pod \"must-gather-qk869\" (UID: \"cc1f65e4-2649-4f0c-828d-2e6699fbd207\") " pod="openshift-must-gather-df522/must-gather-qk869"
Sep 11 17:06:53.732141 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.732121    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cc1f65e4-2649-4f0c-828d-2e6699fbd207-must-gather-output\") pod \"must-gather-qk869\" (UID: \"cc1f65e4-2649-4f0c-828d-2e6699fbd207\") " pod="openshift-must-gather-df522/must-gather-qk869"
Sep 11 17:06:53.748188 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.748158    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms7jw\" (UniqueName: \"kubernetes.io/projected/cc1f65e4-2649-4f0c-828d-2e6699fbd207-kube-api-access-ms7jw\") pod \"must-gather-qk869\" (UID: \"cc1f65e4-2649-4f0c-828d-2e6699fbd207\") " pod="openshift-must-gather-df522/must-gather-qk869"
Sep 11 17:06:53.879625 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:53.879541    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-df522/must-gather-qk869"
Sep 11 17:06:54.186054 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:54.186029    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-df522/must-gather-qk869"]
Sep 11 17:06:54.188033 ip-10-0-135-58 kubenswrapper[2545]: W0911 17:06:54.188006    2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc1f65e4_2649_4f0c_828d_2e6699fbd207.slice/crio-24434ff78fcfbac4375784fb26ff91c9bf467057c187902f9ba57fcee6ea4c8a WatchSource:0}: Error finding container 24434ff78fcfbac4375784fb26ff91c9bf467057c187902f9ba57fcee6ea4c8a: Status 404 returned error can't find the container with id 24434ff78fcfbac4375784fb26ff91c9bf467057c187902f9ba57fcee6ea4c8a
Sep 11 17:06:54.192973 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:54.192943    2545 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:06:54.547419 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:54.547379    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-df522/must-gather-qk869" event={"ID":"cc1f65e4-2649-4f0c-828d-2e6699fbd207","Type":"ContainerStarted","Data":"24434ff78fcfbac4375784fb26ff91c9bf467057c187902f9ba57fcee6ea4c8a"}
Sep 11 17:06:55.557797 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:55.557750    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-df522/must-gather-qk869" event={"ID":"cc1f65e4-2649-4f0c-828d-2e6699fbd207","Type":"ContainerStarted","Data":"8757afb20267ff4d3f1b3f43130e769c3f367f937b4620c1bb70f1b5baf3ec1e"}
Sep 11 17:06:55.558319 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:55.557823    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-df522/must-gather-qk869" event={"ID":"cc1f65e4-2649-4f0c-828d-2e6699fbd207","Type":"ContainerStarted","Data":"ea0dde4305f6b9f7f487e38c7fc7c54e1ffc53ee9d7003ca3b1f7e3a260461a4"}
Sep 11 17:06:55.577187 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:55.577125    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-df522/must-gather-qk869" podStartSLOduration=1.819026853 podStartE2EDuration="2.577104419s" totalImagesPullingTime="758.077566ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:06:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:06:55.574025794 +0000 UTC m=+2234.127195756" watchObservedRunningTime="2026-09-11 17:06:55.577104419 +0000 UTC m=+2234.130274348"
Sep 11 17:06:56.716558 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:56.716521    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wfmld_cf6faf2b-abcc-4abe-bf4f-d7496123108c/global-pull-secret-syncer/0.log"
Sep 11 17:06:56.939129 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:56.939086    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tzdpv_84351f8e-eeb4-43fc-9e43-465096bcf573/konnectivity-agent/0.log"
Sep 11 17:06:56.998179 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:06:56.998148    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-58.ec2.internal_59c79f2c70829d40ee96e19328eda4a1/haproxy/0.log"
Sep 11 17:07:01.599378 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:01.599340    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-q75xn_049a553b-8d4a-4376-8b00-9950b7db7b61/manager/0.log"
Sep 11 17:07:01.635267 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:01.635232    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-qgz2d_3df23723-01c6-4388-9708-8179b50c71fb/limitador/0.log"
Sep 11 17:07:03.355775 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:03.355739    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_809e1548-970a-4190-8bd9-90ab402c1e50/alertmanager/0.log"
Sep 11 17:07:03.405121 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:03.405028    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_809e1548-970a-4190-8bd9-90ab402c1e50/config-reloader/0.log"
Sep 11 17:07:03.439960 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:03.439931    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_809e1548-970a-4190-8bd9-90ab402c1e50/kube-rbac-proxy-web/0.log"
Sep 11 17:07:03.488225 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:03.488184    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_809e1548-970a-4190-8bd9-90ab402c1e50/kube-rbac-proxy/0.log"
Sep 11 17:07:03.546389 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:03.546312    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_809e1548-970a-4190-8bd9-90ab402c1e50/kube-rbac-proxy-metric/0.log"
Sep 11 17:07:03.584329 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:03.584293    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_809e1548-970a-4190-8bd9-90ab402c1e50/prom-label-proxy/0.log"
Sep 11 17:07:03.618137 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:03.618046    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_809e1548-970a-4190-8bd9-90ab402c1e50/init-config-reloader/0.log"
Sep 11 17:07:03.792619 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:03.792590    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-697b76879d-wxkfr_081b0f85-0ef1-455b-8328-9ca132d8907b/metrics-server/0.log"
Sep 11 17:07:03.831667 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:03.831636    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-k4758_5b7f295c-29f6-4c32-bd02-16ccc91a2ba5/monitoring-plugin/0.log"
Sep 11 17:07:04.081528 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:04.081499    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c6vf4_d771d5d4-dde8-4dd0-90ee-999f015e2a96/node-exporter/0.log"
Sep 11 17:07:04.108788 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:04.108756    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c6vf4_d771d5d4-dde8-4dd0-90ee-999f015e2a96/kube-rbac-proxy/0.log"
Sep 11 17:07:04.135307 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:04.135273    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c6vf4_d771d5d4-dde8-4dd0-90ee-999f015e2a96/init-textfile/0.log"
Sep 11 17:07:04.505412 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:04.505378    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-nrbh4_7a4a59db-577c-4520-b770-71c8b0bca905/prometheus-operator/0.log"
Sep 11 17:07:04.534340 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:04.534315    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-nrbh4_7a4a59db-577c-4520-b770-71c8b0bca905/kube-rbac-proxy/0.log"
Sep 11 17:07:04.611957 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:04.611926    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66649b79bd-4jr7t_cf6691ec-f9d1-4e88-b861-23e617102881/telemeter-client/0.log"
Sep 11 17:07:04.645024 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:04.644981    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66649b79bd-4jr7t_cf6691ec-f9d1-4e88-b861-23e617102881/reload/0.log"
Sep 11 17:07:04.678591 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:04.678558    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66649b79bd-4jr7t_cf6691ec-f9d1-4e88-b861-23e617102881/kube-rbac-proxy/0.log"
Sep 11 17:07:06.275283 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.275240    2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"]
Sep 11 17:07:06.281085 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.281057    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.287408 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.287378    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"]
Sep 11 17:07:06.368486 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.368445    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swpd5\" (UniqueName: \"kubernetes.io/projected/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-kube-api-access-swpd5\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.368689 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.368555    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-proc\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.368768 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.368690    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-lib-modules\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.368768 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.368731    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-podres\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.368939 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.368906    2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-sys\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.469697 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.469655    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-sys\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.469934 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.469747    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swpd5\" (UniqueName: \"kubernetes.io/projected/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-kube-api-access-swpd5\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.469934 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.469792    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-proc\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.469934 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.469808    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-sys\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.469934 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.469917    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-lib-modules\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.470163 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.469947    2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-podres\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.470163 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.469919    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-proc\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.470163 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.470029    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-lib-modules\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.470163 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.470071    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-podres\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.480538 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.480513    2545 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swpd5\" (UniqueName: \"kubernetes.io/projected/b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1-kube-api-access-swpd5\") pod \"perf-node-gather-daemonset-hq5ln\" (UID: \"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1\") " pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.595407 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.595329    2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:06.790640 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.790600    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/1.log"
Sep 11 17:07:06.799856 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.799736    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-vbjs6_c10438b1-8196-417e-a982-d7c12e9cdb20/console-operator/2.log"
Sep 11 17:07:06.895477 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:06.895376    2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"]
Sep 11 17:07:07.657173 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:07.657138    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln" event={"ID":"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1","Type":"ContainerStarted","Data":"eefb6348548750cf2a52a259ed39ac42ba4dcde388dbb153ecf825c99606cc9d"}
Sep 11 17:07:07.657577 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:07.657219    2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln" event={"ID":"b977cfd1-e3f8-4a4f-a5f9-50da8fce6bb1","Type":"ContainerStarted","Data":"89ac655130a9ae5971fa3e3c23eaf1bf68bf6de4ce12f9ed9ee1b6c3345dcba5"}
Sep 11 17:07:07.657577 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:07.657263    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:07.676681 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:07.676621    2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln" podStartSLOduration=1.6766011619999999 podStartE2EDuration="1.676601162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:07:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:07:07.673090176 +0000 UTC m=+2246.226260124" watchObservedRunningTime="2026-09-11 17:07:07.676601162 +0000 UTC m=+2246.229771089"
Sep 11 17:07:08.162453 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:08.162419    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-llqgw_8cb42ba7-0e54-493a-ba9a-9ab775943b15/volume-data-source-validator/0.log"
Sep 11 17:07:09.314673 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:09.314637    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m4w4b_cb3167ad-e92f-4c98-8e23-f68f3f46e399/dns/0.log"
Sep 11 17:07:09.340377 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:09.340335    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m4w4b_cb3167ad-e92f-4c98-8e23-f68f3f46e399/kube-rbac-proxy/0.log"
Sep 11 17:07:09.494766 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:09.494738    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lxlpg_9350ed8f-0176-49c6-b2c7-92ee69b2c79a/dns-node-resolver/0.log"
Sep 11 17:07:10.154693 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:10.154659    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-58957684c5-2wtmf_71509d28-451f-48e2-881f-b1eb1030411c/registry/0.log"
Sep 11 17:07:10.185725 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:10.185696    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8vt9w_5415fcf3-0701-4c3a-8582-6a614a28ab4a/node-ca/0.log"
Sep 11 17:07:11.218630 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:11.218598    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p_92d8390b-90e0-4d31-ac76-b6496a647f85/istio-proxy/0.log"
Sep 11 17:07:12.656922 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:12.656896    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dpm7g_7716e16f-2459-4ab2-a157-44b83117f9c4/serve-healthcheck-canary/0.log"
Sep 11 17:07:13.442648 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:13.442617    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rm228_616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4/kube-rbac-proxy/0.log"
Sep 11 17:07:13.471793 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:13.471769    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rm228_616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4/exporter/0.log"
Sep 11 17:07:13.502660 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:13.502627    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rm228_616ec0d0-c00e-4bd8-8590-c56b1a2ba0a4/extractor/0.log"
Sep 11 17:07:13.676522 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:13.676493    2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-df522/perf-node-gather-daemonset-hq5ln"
Sep 11 17:07:16.093119 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:16.093093    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bhszh_47711732-e59c-4fe5-8650-f8f97b126a14/manager/0.log"
Sep 11 17:07:16.562526 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:16.562496    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-j98b2_10039705-5220-481d-8445-a97e9eb6285d/server/0.log"
Sep 11 17:07:16.736974 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:16.736947    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-trq6d_381b5264-ef15-4c99-9e2c-924f70fed40f/manager/0.log"
Sep 11 17:07:16.967689 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:16.967657    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665d69bffd-dpbh5_f2a8594c-3871-41a0-b96f-73b0c0175bfa/manager/0.log"
Sep 11 17:07:16.995949 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:16.995888    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665d69bffd-dpbh5_f2a8594c-3871-41a0-b96f-73b0c0175bfa/copy-manifests/0.log"
Sep 11 17:07:17.006449 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:17.006420    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p_92d8390b-90e0-4d31-ac76-b6496a647f85/istio-proxy/0.log"
Sep 11 17:07:18.419471 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:18.419378    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5bd998d9c-9t57f_eb4c0c97-24db-4e12-8ae0-1585f4cd7df1/manager/0.log"
Sep 11 17:07:18.493026 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:18.492979    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-g24qt_373ff2f4-afd2-4050-a55a-407b14bd4537/openshift-lws-operator/0.log"
Sep 11 17:07:18.902453 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:18.902412    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2rv9p_92d8390b-90e0-4d31-ac76-b6496a647f85/istio-proxy/0.log"
Sep 11 17:07:23.698258 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:23.698220    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-hxkk2_28d927ce-d08b-478a-8cfa-88a8bcfa3159/migrator/0.log"
Sep 11 17:07:23.724848 ip-10-0-135-58 kubenswrapper[2545]: I0911 17:07:23.724821    2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-hxkk2_28d927ce-d08b-478a-8cfa-88a8bcfa3159/graceful-termination/0.log"