Sep 10 13:18:09.062850 ip-10-0-135-117 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 13:18:09.475229 ip-10-0-135-117 kubenswrapper[2551]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 13:18:09.475229 ip-10-0-135-117 kubenswrapper[2551]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 13:18:09.475229 ip-10-0-135-117 kubenswrapper[2551]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 13:18:09.475229 ip-10-0-135-117 kubenswrapper[2551]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 13:18:09.475821 ip-10-0-135-117 kubenswrapper[2551]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 13:18:09.475821 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.475289    2551 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 13:18:09.483617 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483602    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:18:09.483617 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483616    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483620    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483623    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483626    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483629    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483635    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483639    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483642    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483645    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483648    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483651    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483654    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483658    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483662    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483665    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483668    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483676    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483679    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483682    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:18:09.483686 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483684    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483687    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483690    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483693    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483696    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483700    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483703    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483706    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483709    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483712    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483714    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483717    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483720    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483722    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483725    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483727    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483730    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483732    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483735    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483737    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:18:09.484227 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483740    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483743    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483746    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483748    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483751    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483753    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483757    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483759    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483762    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483764    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483767    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483769    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483772    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483774    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483778    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483781    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483783    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483786    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483789    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483791    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:18:09.484959 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483794    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483796    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483799    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483802    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483804    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483807    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483809    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483812    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483815    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483817    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483820    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483822    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483825    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483827    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483829    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483833    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483835    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483838    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483840    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483843    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:18:09.485480 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483846    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483848    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483852    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483855    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483857    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483860    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483863    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483866    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483870    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483874    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483877    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483880    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483882    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483885    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483888    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483891    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483894    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483896    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:18:09.486018 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.483899    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:18:09.486740 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486726    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:18:09.486740 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486740    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486744    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486747    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486750    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486753    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486755    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486758    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486761    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486763    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486766    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486768    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486771    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486774    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486776    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486779    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486781    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486785    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486787    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486790    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486793    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:18:09.486802 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486796    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486798    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486801    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486804    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486807    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486809    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486819    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486826    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486829    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486832    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486834    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486837    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486839    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486842    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486845    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486847    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486849    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486852    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486854    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486857    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:18:09.487420 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486859    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486862    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486865    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486867    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486870    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486872    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486875    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486877    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486880    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486882    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486884    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486887    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486889    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486892    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486896    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486898    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486901    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486905    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486907    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486912    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:18:09.487971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486916    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486920    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486922    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486926    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486930    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486933    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486936    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486940    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486943    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486945    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486948    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486950    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486953    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486955    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486958    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486960    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486963    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486966    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486968    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:18:09.488458 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486971    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486973    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486976    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486979    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486982    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486985    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486988    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486991    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486993    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.486996    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487000    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487005    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487008    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487010    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487013    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487015    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487018    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487020    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487023    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:18:09.488945 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487410    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487416    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487420    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487423    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487426    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487429    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487432    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487435    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487439    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487441    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487444    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487447    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487450    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487453    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487455    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487458    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487461    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487464    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487467    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:18:09.489414 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487470    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487473    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487476    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487478    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487481    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487483    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487485    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487488    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487490    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487493    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487495    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487498    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487500    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487503    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487505    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487508    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487510    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487513    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487516    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487519    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:18:09.489893 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487521    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487525    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487528    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487531    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487533    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487535    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487538    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487540    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487544    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487547    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487551    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487553    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487559    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487562    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487565    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487568    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487586    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487589    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487594    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:18:09.490368 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487597    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487600    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487603    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487606    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487609    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487612    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487614    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487617    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487620    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487622    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487625    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487628    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487631    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487633    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487636    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487638    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487641    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487644    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487646    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487649    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:18:09.490841 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487651    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487654    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487656    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487660    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487663    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487667    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487670    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487673    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487676    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487678    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487681    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487683    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487686    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487688    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487691    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487693    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487696    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487699    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487702    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487704    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:18:09.491331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.487706    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487776    2551 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487809    2551 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487814    2551 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487818    2551 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487823    2551 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487826    2551 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487830    2551 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487834    2551 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487837    2551 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487840    2551 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487844    2551 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487847    2551 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487850    2551 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487853    2551 flags.go:64] FLAG: --cgroup-root=""
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487856    2551 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487859    2551 flags.go:64] FLAG: --client-ca-file=""
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487864    2551 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487867    2551 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487873    2551 flags.go:64] FLAG: --cluster-domain=""
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487875    2551 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487878    2551 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487881    2551 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487885    2551 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 13:18:09.491843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487888    2551 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487891    2551 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487894    2551 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487897    2551 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487900    2551 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487903    2551 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487905    2551 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487908    2551 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487911    2551 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487915    2551 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487918    2551 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487920    2551 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487923    2551 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487926    2551 flags.go:64] FLAG: --enable-server="true"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487929    2551 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487933    2551 flags.go:64] FLAG: --event-burst="100"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487936    2551 flags.go:64] FLAG: --event-qps="50"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487939    2551 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487942    2551 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487944    2551 flags.go:64] FLAG: --eviction-hard=""
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487948    2551 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487951    2551 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487954    2551 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487956    2551 flags.go:64] FLAG: --eviction-soft=""
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487959    2551 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 13:18:09.492426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487963    2551 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487969    2551 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487972    2551 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487975    2551 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487978    2551 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487980    2551 flags.go:64] FLAG: --feature-gates=""
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487984    2551 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487987    2551 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487991    2551 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487994    2551 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.487997    2551 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488000    2551 flags.go:64] FLAG: --help="false"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488003    2551 flags.go:64] FLAG: --hostname-override="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488006    2551 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488009    2551 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488012    2551 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488016    2551 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488019    2551 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488022    2551 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488025    2551 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488028    2551 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488030    2551 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488033    2551 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488036    2551 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 13:18:09.493087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488039    2551 flags.go:64] FLAG: --kube-reserved=""
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488042    2551 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488045    2551 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488048    2551 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488050    2551 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488053    2551 flags.go:64] FLAG: --lock-file=""
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488056    2551 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488059    2551 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488064    2551 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488070    2551 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488074    2551 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488076    2551 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488079    2551 flags.go:64] FLAG: --logging-format="text"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488082    2551 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488086    2551 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488089    2551 flags.go:64] FLAG: --manifest-url=""
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488091    2551 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488095    2551 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488098    2551 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488103    2551 flags.go:64] FLAG: --max-pods="110"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488106    2551 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488108    2551 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488111    2551 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488114    2551 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488117    2551 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 13:18:09.493703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488120    2551 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488123    2551 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488130    2551 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488133    2551 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488136    2551 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488139    2551 flags.go:64] FLAG: --pod-cidr=""
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488141    2551 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488147    2551 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488149    2551 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488152    2551 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488155    2551 flags.go:64] FLAG: --port="10250"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488158    2551 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488161    2551 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-015ba32dae5e39806"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488164    2551 flags.go:64] FLAG: --qos-reserved=""
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488167    2551 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488172    2551 flags.go:64] FLAG: --register-node="true"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488175    2551 flags.go:64] FLAG: --register-with-taints=""
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488180    2551 flags.go:64] FLAG: --registry-burst="10"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488183    2551 flags.go:64] FLAG: --registry-qps="5"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488186    2551 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488189    2551 flags.go:64] FLAG: --reserved-memory=""
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488192    2551 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488195    2551 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488198    2551 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488201    2551 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 13:18:09.494313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488204    2551 flags.go:64] FLAG: --runonce="false"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488207    2551 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488210    2551 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488213    2551 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488215    2551 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488218    2551 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488222    2551 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488225    2551 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488227    2551 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488230    2551 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488233    2551 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488236    2551 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488239    2551 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488242    2551 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488245    2551 flags.go:64] FLAG: --system-cgroups=""
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488248    2551 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488253    2551 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488256    2551 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488258    2551 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488263    2551 flags.go:64] FLAG: --tls-min-version=""
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488265    2551 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488268    2551 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488272    2551 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488275    2551 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488279    2551 flags.go:64] FLAG: --v="2"
Sep 10 13:18:09.494952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488283    2551 flags.go:64] FLAG: --version="false"
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488287    2551 flags.go:64] FLAG: --vmodule=""
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488290    2551 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.488293    2551 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488406    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488410    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488413    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488416    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488419    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488422    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488424    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488427    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488430    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488433    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488436    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488438    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488441    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488445    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488448    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:18:09.495615 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488452    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488455    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488457    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488460    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488464    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488467    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488470    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488473    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488476    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488479    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488482    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488485    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488488    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488491    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488493    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488496    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488499    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488501    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488504    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:18:09.496090 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488507    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488509    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488512    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488514    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488517    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488519    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488522    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488524    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488526    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488529    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488532    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488534    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488537    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488539    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488542    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488544    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488547    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488549    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488552    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488554    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:18:09.496552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488557    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488560    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488562    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488565    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488567    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488584    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488587    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488589    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488592    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488595    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488598    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488601    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488603    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488606    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488609    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488613    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488616    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488619    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488622    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488625    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:18:09.497088 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488628    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488630    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488633    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488635    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488638    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488640    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488643    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488646    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488648    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488651    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488653    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488656    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488659    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488661    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488664    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488666    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488669    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488673    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488676    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488678    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:18:09.497647 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488681    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488683    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488686    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488690    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.488692    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.489594    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.496781    2551 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.496797    2551 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.497758    2551 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.497770    2551 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497825    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497830    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497833    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497836    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497839    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:18:09.498129 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497842    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497845    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497848    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497850    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497853    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497856    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497858    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497861    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497864    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497867    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497869    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497872    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497874    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497877    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497880    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497882    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497885    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497888    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497890    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497893    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:18:09.498512 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497895    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497898    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497900    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497903    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497907    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497910    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497912    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497915    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497917    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497920    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497922    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497925    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497927    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497930    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497932    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497935    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497937    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497940    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497942    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497945    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:18:09.499022 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497947    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497951    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497954    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497957    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497960    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497962    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497965    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497967    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497970    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497973    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497975    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497978    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497980    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497983    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497985    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497988    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497991    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497994    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497997    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:18:09.499504 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.497999    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498002    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498004    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498007    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498009    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498012    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498014    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498017    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498019    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498022    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498024    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498027    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498029    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498032    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498035    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498037    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498040    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498042    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498047    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498050    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:18:09.499989 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498053    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498056    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498059    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498062    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498066    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498068    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498071    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498073    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498077    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498080    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498082    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498086    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498089    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498091    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498094    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:18:09.500493 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.498099    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498204    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498208    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498211    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498214    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498216    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498219    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498222    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498224    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498227    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498229    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498233    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498237    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498241    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498244    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498247    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498250    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498253    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:18:09.500928 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498256    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498259    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498261    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498264    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498267    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498269    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498272    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498274    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498277    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498280    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498282    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498285    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498288    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498291    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498294    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498296    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498299    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498301    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498304    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498306    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:18:09.501383 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498309    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498311    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498313    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498316    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498318    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498321    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498323    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498326    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498328    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498331    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498333    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498336    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498338    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498341    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498343    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498346    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498348    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498351    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498353    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498356    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:18:09.501940 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498358    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498361    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498363    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498366    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498369    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498371    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498374    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498377    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498379    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498381    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498384    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498386    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498389    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498391    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498394    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498396    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498399    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498401    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498404    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498406    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:18:09.502428 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498409    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498411    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498414    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498417    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498419    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498422    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498425    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498428    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498431    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498433    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498436    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498439    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498441    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498447    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498449    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498452    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498454    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498457    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498461    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:18:09.502924 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498463    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:18:09.503373 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498466    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:18:09.503373 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:09.498468    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:18:09.503373 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.498473    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 13:18:09.503373 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.498596    2551 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 13:18:09.503373 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.500607    2551 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 13:18:09.503373 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.501501    2551 server.go:1019] "Starting client certificate rotation"
Sep 10 13:18:09.503373 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.501631    2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 13:18:09.503373 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.501666    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 13:18:09.504161 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.504149    2551 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 13:18:09.507956 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.507942    2551 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 13:18:09.525013 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.524983    2551 certificate_manager.go:596] "Failed while requesting a signed certificate from the control plane" err="cannot create certificate signing request: certificatesigningrequests.certificates.k8s.io is forbidden: User \"system:serviceaccount:openshift-machine-config-operator:node-bootstrapper\" cannot create resource \"certificatesigningrequests\" in API group \"certificates.k8s.io\" at the cluster scope" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError"
Sep 10 13:18:09.525973 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.525959    2551 log.go:25] "Validated CRI v1 runtime API"
Sep 10 13:18:09.530707 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.530690    2551 log.go:25] "Validated CRI v1 image API"
Sep 10 13:18:09.531462 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.531450    2551 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 13:18:09.534802 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.534784    2551 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 82a78580-b0f8-4b96-98db-e5b0975b65f8:/dev/nvme0n1p3 c281309f-566d-488a-b920-2ba74df52855:/dev/nvme0n1p4]
Sep 10 13:18:09.534847 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.534802    2551 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 13:18:09.541297 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.541181    2551 manager.go:216] Machine: {Timestamp:2026-09-10 13:18:09.539833751 +0000 UTC m=+0.358872142 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3092838 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27e6e9ed9215d57737692902c3c445 SystemUUID:ec27e6e9-ed92-15d5-7737-692902c3c445 BootID:bc868f5d-803e-4226-8d03-fcb8e44b3b69 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:43:3c:8f:78:0d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:43:3c:8f:78:0d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:39:e5:a5:cc:0d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 13:18:09.541297 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.541289    2551 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 13:18:09.541433 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.541359    2551 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 13:18:09.545377 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.545345    2551 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 13:18:09.545506 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.545379    2551 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-117.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 13:18:09.546229 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.546219    2551 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 13:18:09.546262 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.546232    2551 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 13:18:09.546262 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.546239    2551 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 13:18:09.546870 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.546859    2551 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 13:18:09.548186 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.548176    2551 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 13:18:09.551168 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.551157    2551 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 13:18:09.551278 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.551268    2551 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 13:18:09.551309 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.551302    2551 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 13:18:09.551342 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.551314    2551 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 13:18:09.551342 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.551325    2551 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 13:18:09.551342 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.551334    2551 kubelet.go:398] "Adding apiserver pod source"
Sep 10 13:18:09.551342 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.551343    2551 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 13:18:09.551661 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.551650    2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 13:18:09.551697 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.551668    2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 13:18:09.554215 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.554199    2551 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 13:18:09.556220 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.556206    2551 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 13:18:09.556302 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.556236    2551 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 13:18:09.556428 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.556417    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 13:18:09.556461 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.556434    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 13:18:09.556461 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.556440    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 13:18:09.556461 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.556446    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 13:18:09.556461 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.556457    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 13:18:09.556561 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.556466    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 13:18:09.557017 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.557008    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 13:18:09.557017 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.557017    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 13:18:09.557078 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.557023    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 13:18:09.557078 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.557028    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 13:18:09.557078 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.557043    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 13:18:09.557078 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.557050    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 13:18:09.558988 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.558974    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 13:18:09.558988 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.558985    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 13:18:09.562438 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.562408    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:09.562552 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.562530    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 13:18:09.562728 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.562711    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 13:18:09.562764 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.562757    2551 server.go:1268] "Started kubelet"
Sep 10 13:18:09.562967 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.562883    2551 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 13:18:09.563164 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.563006    2551 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 13:18:09.563229 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.563210    2551 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 13:18:09.564617 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.564587    2551 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 13:18:09.564749 ip-10-0-135-117 systemd[1]: Started Kubernetes Kubelet.
Sep 10 13:18:09.566486 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.566468    2551 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 13:18:09.567343 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.567323    2551 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 13:18:09.569561 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.569541    2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 13:18:09.570253 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.570236    2551 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 13:18:09.570539 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.570518    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:18:09.571183 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.571166    2551 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 13:18:09.571183 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.571168    2551 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 13:18:09.571318 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.571194    2551 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 13:18:09.571318 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.571246    2551 factory.go:153] Registering CRI-O factory
Sep 10 13:18:09.571318 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.571263    2551 factory.go:223] Registration of the crio container factory successfully
Sep 10 13:18:09.571460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.571317    2551 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 13:18:09.571460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.571331    2551 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 13:18:09.571460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.571322    2551 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 13:18:09.571460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.571353    2551 factory.go:55] Registering systemd factory
Sep 10 13:18:09.571460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.571362    2551 factory.go:223] Registration of the systemd container factory successfully
Sep 10 13:18:09.571460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.571386    2551 factory.go:103] Registering Raw factory
Sep 10 13:18:09.571460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.571440    2551 manager.go:1196] Started watching for new ooms in manager
Sep 10 13:18:09.572071 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.572055    2551 manager.go:349] Starting recovery of all containers
Sep 10 13:18:09.576718 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.571960    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f201e01b0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.562730928 +0000 UTC m=+0.381769318,LastTimestamp:2026-09-10 13:18:09.562730928 +0000 UTC m=+0.381769318,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.583333 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.583306    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 13:18:09.583333 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.583319    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 13:18:09.587613 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.587490    2551 manager.go:354] Recovery completed
Sep 10 13:18:09.592141 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.592129    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:09.593691 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.593668    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:09.593775 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.593708    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:09.593775 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.593723    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:09.594268 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.594251    2551 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 13:18:09.594314 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.594267    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:09.594360 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.594268    2551 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 13:18:09.594390 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.594372    2551 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 13:18:09.594959 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.594943    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:09.595003 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.594970    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:09.595003 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.594979    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:09.595721 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.595709    2551 policy_none.go:49] "None policy: Start"
Sep 10 13:18:09.595721 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.595661    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f66f69  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593692009 +0000 UTC m=+0.412730405,LastTimestamp:2026-09-10 13:18:09.593692009 +0000 UTC m=+0.412730405,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.595847 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.595725    2551 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 13:18:09.595847 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.595735    2551 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 13:18:09.596735 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.596720    2551 policy_none.go:47] "Start"
Sep 10 13:18:09.603992 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.603910    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593714415 +0000 UTC m=+0.412752808,LastTimestamp:2026-09-10 13:18:09.593714415 +0000 UTC m=+0.412752808,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.613717 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.613637    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6fc60  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593728096 +0000 UTC m=+0.412766488,LastTimestamp:2026-09-10 13:18:09.593728096 +0000 UTC m=+0.412766488,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.641271 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.625152    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f66f69\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f66f69  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593692009 +0000 UTC m=+0.412730405,LastTimestamp:2026-09-10 13:18:09.594958779 +0000 UTC m=+0.413997169,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.641271 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.636112    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593714415 +0000 UTC m=+0.412752808,LastTimestamp:2026-09-10 13:18:09.594974991 +0000 UTC m=+0.414013382,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.641271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.636177    2551 manager.go:337] "Starting Device Plugin manager"
Sep 10 13:18:09.641271 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.636220    2551 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 13:18:09.641271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.636230    2551 server.go:87] "Starting device plugin registration server"
Sep 10 13:18:09.641271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.636503    2551 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 13:18:09.641271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.636513    2551 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 13:18:09.641271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.636618    2551 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 13:18:09.641271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.636685    2551 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 13:18:09.641271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.636691    2551 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 13:18:09.641271 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.637180    2551 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 13:18:09.641271 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.637232    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:18:09.651227 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.651170    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f6fc60\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6fc60  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593728096 +0000 UTC m=+0.412766488,LastTimestamp:2026-09-10 13:18:09.594983121 +0000 UTC m=+0.414021512,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.660231 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.660163    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f24a1e2e8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.638482664 +0000 UTC m=+0.457521042,LastTimestamp:2026-09-10 13:18:09.638482664 +0000 UTC m=+0.457521042,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.702287 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.702266    2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 13:18:09.703364 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.703339    2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 13:18:09.703451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.703386    2551 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 13:18:09.703451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.703408    2551 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 13:18:09.703553 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.703468    2551 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 13:18:09.715507 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.715488    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 10 13:18:09.736667 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.736614    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:09.737364 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.737347    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:09.737458 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.737379    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:09.737458 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.737394    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:09.737458 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.737422    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.747711 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.747651    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f66f69\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f66f69  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593692009 +0000 UTC m=+0.412730405,LastTimestamp:2026-09-10 13:18:09.737362772 +0000 UTC m=+0.556401166,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.749845 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.749779    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593714415 +0000 UTC m=+0.412752808,LastTimestamp:2026-09-10 13:18:09.737385649 +0000 UTC m=+0.556424042,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.749941 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.749869    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.755972 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.755914    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f6fc60\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6fc60  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593728096 +0000 UTC m=+0.412766488,LastTimestamp:2026-09-10 13:18:09.737399386 +0000 UTC m=+0.556437779,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.786029 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.786005    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms"
Sep 10 13:18:09.804132 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.804106    2551 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-117.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal"]
Sep 10 13:18:09.804191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.804165    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:09.805535 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.805521    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:09.805633 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.805551    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:09.805633 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.805566    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:09.807321 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.807302    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:09.807428 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.807377    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.807428 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.807425    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:09.808344 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.808328    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:09.808498 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.808356    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:09.808498 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.808371    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:09.808498 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.808414    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:09.808498 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.808443    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:09.808498 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.808458    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:09.810914 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.810894    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.810996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.810934    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:09.811610 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.811591    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:09.811693 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.811624    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:09.811693 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.811637    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:09.814432 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.814355    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f66f69\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f66f69  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593692009 +0000 UTC m=+0.412730405,LastTimestamp:2026-09-10 13:18:09.805535665 +0000 UTC m=+0.624574063,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.823088 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.823070    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.824897 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.824845    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593714415 +0000 UTC m=+0.412752808,LastTimestamp:2026-09-10 13:18:09.805558391 +0000 UTC m=+0.624596783,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.827207 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.827191    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.834102 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.834047    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f6fc60\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6fc60  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593728096 +0000 UTC m=+0.412766488,LastTimestamp:2026-09-10 13:18:09.805585198 +0000 UTC m=+0.624623591,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.842493 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.842441    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f66f69\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f66f69  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593692009 +0000 UTC m=+0.412730405,LastTimestamp:2026-09-10 13:18:09.808343318 +0000 UTC m=+0.627381711,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.855124 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.855065    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593714415 +0000 UTC m=+0.412752808,LastTimestamp:2026-09-10 13:18:09.808362759 +0000 UTC m=+0.627401153,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.867379 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.867326    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f6fc60\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6fc60  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593728096 +0000 UTC m=+0.412766488,LastTimestamp:2026-09-10 13:18:09.80838194 +0000 UTC m=+0.627420341,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.873358 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.873334    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3508d0a07aa0fd49c57a7f8e3fd46334-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal\" (UID: \"3508d0a07aa0fd49c57a7f8e3fd46334\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.873410 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.873359    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4a08d7fc3d09b5f1580b5cd035f70a19-config\") pod \"kube-apiserver-proxy-ip-10-0-135-117.ec2.internal\" (UID: \"4a08d7fc3d09b5f1580b5cd035f70a19\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.873410 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.873374    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3508d0a07aa0fd49c57a7f8e3fd46334-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal\" (UID: \"3508d0a07aa0fd49c57a7f8e3fd46334\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.878822 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.878771    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f66f69\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f66f69  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593692009 +0000 UTC m=+0.412730405,LastTimestamp:2026-09-10 13:18:09.808425792 +0000 UTC m=+0.627464184,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.889480 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.889424    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593714415 +0000 UTC m=+0.412752808,LastTimestamp:2026-09-10 13:18:09.808449477 +0000 UTC m=+0.627487870,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.900964 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.900910    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f6fc60\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6fc60  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593728096 +0000 UTC m=+0.412766488,LastTimestamp:2026-09-10 13:18:09.808464135 +0000 UTC m=+0.627502528,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.909804 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.909743    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f66f69\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f66f69  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593692009 +0000 UTC m=+0.412730405,LastTimestamp:2026-09-10 13:18:09.811608955 +0000 UTC m=+0.630647350,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.918636 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.918557    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593714415 +0000 UTC m=+0.412752808,LastTimestamp:2026-09-10 13:18:09.81163089 +0000 UTC m=+0.630669280,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.928074 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.928015    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f6fc60\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6fc60  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593728096 +0000 UTC m=+0.412766488,LastTimestamp:2026-09-10 13:18:09.811640581 +0000 UTC m=+0.630678972,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.950177 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.950153    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:09.950952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.950931    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:09.951028 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.950956    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:09.951028 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.950966    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:09.951028 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.950996    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.959629 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.959549    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f66f69\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f66f69  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593692009 +0000 UTC m=+0.412730405,LastTimestamp:2026-09-10 13:18:09.9509444 +0000 UTC m=+0.769982790,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.962364 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.962308    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-117.ec2.internal.18d3f81f21f6c6ef  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-117.ec2.internal,UID:ip-10-0-135-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-117.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:09.593714415 +0000 UTC m=+0.412752808,LastTimestamp:2026-09-10 13:18:09.950960533 +0000 UTC m=+0.769998925,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:09.970304 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:09.970286    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.973651 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.973633    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4a08d7fc3d09b5f1580b5cd035f70a19-config\") pod \"kube-apiserver-proxy-ip-10-0-135-117.ec2.internal\" (UID: \"4a08d7fc3d09b5f1580b5cd035f70a19\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.973704 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.973660    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3508d0a07aa0fd49c57a7f8e3fd46334-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal\" (UID: \"3508d0a07aa0fd49c57a7f8e3fd46334\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.973704 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.973675    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3508d0a07aa0fd49c57a7f8e3fd46334-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal\" (UID: \"3508d0a07aa0fd49c57a7f8e3fd46334\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.973770 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.973713    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3508d0a07aa0fd49c57a7f8e3fd46334-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal\" (UID: \"3508d0a07aa0fd49c57a7f8e3fd46334\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.973770 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.973746    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3508d0a07aa0fd49c57a7f8e3fd46334-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal\" (UID: \"3508d0a07aa0fd49c57a7f8e3fd46334\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal"
Sep 10 13:18:09.973830 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:09.973746    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4a08d7fc3d09b5f1580b5cd035f70a19-config\") pod \"kube-apiserver-proxy-ip-10-0-135-117.ec2.internal\" (UID: \"4a08d7fc3d09b5f1580b5cd035f70a19\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-117.ec2.internal"
Sep 10 13:18:10.125373 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:10.125321    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-117.ec2.internal"
Sep 10 13:18:10.131383 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:10.131369    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal"
Sep 10 13:18:10.188743 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:10.188718    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="800ms"
Sep 10 13:18:10.373601 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:10.373562    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:10.374623 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:10.374607    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:10.374697 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:10.374639    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:10.374697 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:10.374652    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:10.374697 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:10.374679    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:10.394941 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:10.394891    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:10.463250 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:10.463218    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 13:18:10.575434 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:10.575409    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:10.677410 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:10.677351    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:10.678338 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:10.678311    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:10.678434 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:10.678350    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:10.678434 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:10.678364    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:10.760894 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:10.760863    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 10 13:18:10.958572 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:10.958532    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a08d7fc3d09b5f1580b5cd035f70a19.slice/crio-0be3d14f3beff0a1af6f5bc0e51c04695f180c7c0c29bcac9c15fe1ace083661 WatchSource:0}: Error finding container 0be3d14f3beff0a1af6f5bc0e51c04695f180c7c0c29bcac9c15fe1ace083661: Status 404 returned error can't find the container with id 0be3d14f3beff0a1af6f5bc0e51c04695f180c7c0c29bcac9c15fe1ace083661
Sep 10 13:18:10.959011 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:18:10.958993    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3508d0a07aa0fd49c57a7f8e3fd46334.slice/crio-cece7048e2a160aa47f9b393b6bfca177a5beafee7e558b6416c41af5e024c52 WatchSource:0}: Error finding container cece7048e2a160aa47f9b393b6bfca177a5beafee7e558b6416c41af5e024c52: Status 404 returned error can't find the container with id cece7048e2a160aa47f9b393b6bfca177a5beafee7e558b6416c41af5e024c52
Sep 10 13:18:10.964309 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:10.964293    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 13:18:10.977783 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:10.977705    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f81f73ab1463  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\",Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.964485219 +0000 UTC m=+1.783523602,LastTimestamp:2026-09-10 13:18:10.964485219 +0000 UTC m=+1.783523602,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:10.992092 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:10.992065    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="1.6s"
Sep 10 13:18:10.993251 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:10.993216    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 13:18:10.993343 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:10.993207    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-135-117.ec2.internal.18d3f81f73abed64  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-135-117.ec2.internal,UID:4a08d7fc3d09b5f1580b5cd035f70a19,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da130d90f0bffd5c5876c94544c91b2a1be0d23ce2747ddfeafe88e03ee7d8d1\",Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.964540772 +0000 UTC m=+1.783579157,LastTimestamp:2026-09-10 13:18:10.964540772 +0000 UTC m=+1.783579157,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:11.120086 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:11.120056    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 13:18:11.195737 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:11.195714    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:11.196693 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:11.196649    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:11.196693 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:11.196678    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:11.196693 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:11.196693    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:11.196827 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:11.196723    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:11.217357 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:11.217333    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:11.538926 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:11.538892    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 13:18:11.542211 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:11.542190    2551 certificate_manager.go:596] "Failed while requesting a signed certificate from the control plane" err="cannot create certificate signing request: certificatesigningrequests.certificates.k8s.io is forbidden: User \"system:serviceaccount:openshift-machine-config-operator:node-bootstrapper\" cannot create resource \"certificatesigningrequests\" in API group \"certificates.k8s.io\" at the cluster scope" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError"
Sep 10 13:18:11.573282 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:11.573261    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:11.709625 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:11.709543    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" event={"ID":"3508d0a07aa0fd49c57a7f8e3fd46334","Type":"ContainerStarted","Data":"cece7048e2a160aa47f9b393b6bfca177a5beafee7e558b6416c41af5e024c52"}
Sep 10 13:18:11.711647 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:11.711620    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-117.ec2.internal" event={"ID":"4a08d7fc3d09b5f1580b5cd035f70a19","Type":"ContainerStarted","Data":"0be3d14f3beff0a1af6f5bc0e51c04695f180c7c0c29bcac9c15fe1ace083661"}
Sep 10 13:18:12.573417 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:12.573384    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:12.604798 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:12.604765    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="3.2s"
Sep 10 13:18:12.817761 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:12.817724    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:12.819234 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:12.819199    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:12.819370 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:12.819243    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:12.819370 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:12.819256    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:12.819370 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:12.819285    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:12.831661 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:12.831597    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 10 13:18:12.840552 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:12.840514    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:12.886913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:12.886888    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:12.888078 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:12.888056    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:12.888176 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:12.888101    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:12.888176 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:12.888119    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:13.064043 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:13.064000    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 13:18:13.133085 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:13.132999    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-135-117.ec2.internal.18d3f81ff4127383  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-135-117.ec2.internal,UID:4a08d7fc3d09b5f1580b5cd035f70a19,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da130d90f0bffd5c5876c94544c91b2a1be0d23ce2747ddfeafe88e03ee7d8d1\" in 2.154s (2.154s including waiting). Image size: 321671430 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:13.118743427 +0000 UTC m=+3.937781805,LastTimestamp:2026-09-10 13:18:13.118743427 +0000 UTC m=+3.937781805,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:13.145813 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:13.145598    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f81ff416a710  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\" in 2.154s (2.154s including waiting). Image size: 282837706 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:13.119018768 +0000 UTC m=+3.938057154,LastTimestamp:2026-09-10 13:18:13.119018768 +0000 UTC m=+3.938057154,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:13.198789 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:13.198690    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-135-117.ec2.internal.18d3f81ff821e05c  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-135-117.ec2.internal,UID:4a08d7fc3d09b5f1580b5cd035f70a19,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Created,Message:Created container: haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:13.186863196 +0000 UTC m=+4.005901600,LastTimestamp:2026-09-10 13:18:13.186863196 +0000 UTC m=+4.005901600,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:13.210775 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:13.210703    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-135-117.ec2.internal.18d3f81ff892e4e5  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-135-117.ec2.internal,UID:4a08d7fc3d09b5f1580b5cd035f70a19,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Started,Message:Started container haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:13.194269925 +0000 UTC m=+4.013308330,LastTimestamp:2026-09-10 13:18:13.194269925 +0000 UTC m=+4.013308330,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:13.298956 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:13.298920    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 13:18:13.371189 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:13.371085    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f820022ae62c  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:13.355226668 +0000 UTC m=+4.174265071,LastTimestamp:2026-09-10 13:18:13.355226668 +0000 UTC m=+4.174265071,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:13.383751 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:13.383679    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f820029c6928  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:13.362665768 +0000 UTC m=+4.181704166,LastTimestamp:2026-09-10 13:18:13.362665768 +0000 UTC m=+4.181704166,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:13.573105 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:13.573084    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:13.607978 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:13.607953    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 13:18:13.716712 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:13.716637    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" event={"ID":"3508d0a07aa0fd49c57a7f8e3fd46334","Type":"ContainerStarted","Data":"75cb631ff271315d86bc225ab268ddb23e59b7d00093b57126e5248325fd8886"}
Sep 10 13:18:13.716712 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:13.716660    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:13.717622 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:13.717603    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:13.717719 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:13.717632    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:13.717719 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:13.717647    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:13.717880 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:13.717848    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:13.718327 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:13.718308    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-117.ec2.internal" event={"ID":"4a08d7fc3d09b5f1580b5cd035f70a19","Type":"ContainerStarted","Data":"cc167e23e17603b3c5177b4093e9265ff6ba355c23979b0aabf4da0e8dd2c9fd"}
Sep 10 13:18:13.718389 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:13.718351    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:13.719060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:13.719044    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:13.719130 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:13.719072    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:13.719130 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:13.719083    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:13.719256 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:13.719244    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:14.576684 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:14.576643    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:14.719991 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:14.719962    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:14.720114 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:14.719962    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:14.721106 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:14.721086    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:14.721211 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:14.721118    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:14.721211 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:14.721128    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:14.721211 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:14.721086    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:14.721211 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:14.721205    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:14.721409 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:14.721226    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:14.721409 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:14.721325    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:14.721513 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:14.721494    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:15.577342 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:15.577302    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 13:18:15.581818 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:15.581795    2551 certificate_manager.go:596] "Failed while requesting a signed certificate from the control plane" err="cannot create certificate signing request: certificatesigningrequests.certificates.k8s.io is forbidden: User \"system:serviceaccount:openshift-machine-config-operator:node-bootstrapper\" cannot create resource \"certificatesigningrequests\" in API group \"certificates.k8s.io\" at the cluster scope" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError"
Sep 10 13:18:15.583243 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:15.583225    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:15.815453 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:15.815420    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="6.4s"
Sep 10 13:18:16.041436 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:16.041409    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:16.042435 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:16.042416    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:16.042543 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:16.042457    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:16.042543 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:16.042473    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:16.042543 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:16.042507    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:16.060963 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:16.060937    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:16.573109 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:16.573083    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:16.686856 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:16.686824    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 13:18:17.515269 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:17.515233    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 13:18:17.571966 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:17.571940    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:17.651438 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:17.651410    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:17.653334 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:17.653318    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:17.653414 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:17.653348    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:17.653414 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:17.653362    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:18.568020 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:18.567985    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 10 13:18:18.568020 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:18.567998    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:19.445816 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:19.445785    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 13:18:19.572301 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:19.572277    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:19.637716 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:19.637688    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:18:20.571707 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:20.571679    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:21.576184 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:21.576152    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:22.225247 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:22.225218    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Sep 10 13:18:22.461963 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:22.461928    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:22.462921 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:22.462901    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:22.462986 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:22.462936    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:22.462986 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:22.462952    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:22.462986 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:22.462979    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:22.482555 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:22.482504    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:22.571747 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:22.571728    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:23.572528 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:23.572499    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:23.853703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:23.853622    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 13:18:23.856364 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:23.856339    2551 certificate_manager.go:596] "Failed while requesting a signed certificate from the control plane" err="cannot create certificate signing request: certificatesigningrequests.certificates.k8s.io is forbidden: User \"system:serviceaccount:openshift-machine-config-operator:node-bootstrapper\" cannot create resource \"certificatesigningrequests\" in API group \"certificates.k8s.io\" at the cluster scope" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError"
Sep 10 13:18:24.300549 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:24.300519    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 13:18:24.573403 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:24.573344    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:24.736700 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:24.736669    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/0.log"
Sep 10 13:18:24.736805 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:24.736712    2551 generic.go:356] "Generic (PLEG): container finished" podID="3508d0a07aa0fd49c57a7f8e3fd46334" containerID="75cb631ff271315d86bc225ab268ddb23e59b7d00093b57126e5248325fd8886" exitCode=1
Sep 10 13:18:24.736805 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:24.736743    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" event={"ID":"3508d0a07aa0fd49c57a7f8e3fd46334","Type":"ContainerDied","Data":"75cb631ff271315d86bc225ab268ddb23e59b7d00093b57126e5248325fd8886"}
Sep 10 13:18:24.736874 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:24.736837    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:24.737641 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:24.737622    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:24.737686 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:24.737655    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:24.737686 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:24.737666    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:24.737928 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:24.737916    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:24.749240 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:24.749169    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f822a8c94472  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:24.740557938 +0000 UTC m=+15.559596317,LastTimestamp:2026-09-10 13:18:24.740557938 +0000 UTC m=+15.559596317,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:24.826058 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:24.825968    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f820022ae62c\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f820022ae62c  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:13.355226668 +0000 UTC m=+4.174265071,LastTimestamp:2026-09-10 13:18:24.813144207 +0000 UTC m=+15.632182601,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:24.835525 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:24.835462    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f820029c6928\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f820029c6928  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:13.362665768 +0000 UTC m=+4.181704166,LastTimestamp:2026-09-10 13:18:24.81999762 +0000 UTC m=+15.639036011,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:25.572061 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:25.572037    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:25.739338 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:25.739317    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/0.log"
Sep 10 13:18:25.739891 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:25.739358    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" event={"ID":"3508d0a07aa0fd49c57a7f8e3fd46334","Type":"ContainerStarted","Data":"b56f860aaee41849315e936c4fbd75f107e29d60b21506ae24f32f2bfaa5f8a5"}
Sep 10 13:18:25.739891 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:25.739488    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:25.740270 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:25.740253    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:25.740364 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:25.740288    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:25.740364 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:25.740304    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:25.740526 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:25.740512    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:26.572397 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:26.572370    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:26.666877 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:26.666853    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:26.667818 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:26.667793    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:26.667893 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:26.667824    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:26.667893 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:26.667834    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:27.498934 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:27.498908    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 13:18:27.575167 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:27.575146    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:27.742504 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:27.742480    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 13:18:28.572400 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:28.572369    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:29.238905 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:29.238871    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Sep 10 13:18:29.483020 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:29.482995    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:29.484003 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:29.483985    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:29.484088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:29.484017    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:29.484088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:29.484027    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:29.484088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:29.484055    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:29.503915 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:29.503856    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:29.573607 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:29.573587    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:29.637841 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:29.637813    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:18:30.126483 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:30.126455    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 10 13:18:30.571851 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:30.571827    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:31.572825 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:31.572798    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:32.574171 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:32.574143    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:33.573331 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:33.573297    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:34.573789 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:34.573762    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:35.574772 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:35.574748    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:36.247239 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:36.247209    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Sep 10 13:18:36.504528 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.504458    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:36.505395 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.505377    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:36.505497 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.505408    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:36.505497 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.505419    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:36.505497 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.505446    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:36.523349 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:36.523324    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:36.571130 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.571105    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:36.757537 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.757486    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/1.log"
Sep 10 13:18:36.757894 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.757880    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/0.log"
Sep 10 13:18:36.757946 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.757909    2551 generic.go:356] "Generic (PLEG): container finished" podID="3508d0a07aa0fd49c57a7f8e3fd46334" containerID="b56f860aaee41849315e936c4fbd75f107e29d60b21506ae24f32f2bfaa5f8a5" exitCode=1
Sep 10 13:18:36.757946 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.757935    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" event={"ID":"3508d0a07aa0fd49c57a7f8e3fd46334","Type":"ContainerDied","Data":"b56f860aaee41849315e936c4fbd75f107e29d60b21506ae24f32f2bfaa5f8a5"}
Sep 10 13:18:36.758017 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.757973    2551 scope.go:117] "RemoveContainer" containerID="75cb631ff271315d86bc225ab268ddb23e59b7d00093b57126e5248325fd8886"
Sep 10 13:18:36.758057 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.758048    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:36.758848 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.758830    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:36.758932 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.758867    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:36.758932 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.758881    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:36.759269 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:36.759093    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:36.759269 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:36.759151    2551 scope.go:117] "RemoveContainer" containerID="75cb631ff271315d86bc225ab268ddb23e59b7d00093b57126e5248325fd8886"
Sep 10 13:18:36.767414 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:36.767385    2551 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator_3508d0a07aa0fd49c57a7f8e3fd46334_0 in pod sandbox cece7048e2a160aa47f9b393b6bfca177a5beafee7e558b6416c41af5e024c52 from index: no such id: '75cb631ff271315d86bc225ab268ddb23e59b7d00093b57126e5248325fd8886'" containerID="75cb631ff271315d86bc225ab268ddb23e59b7d00093b57126e5248325fd8886"
Sep 10 13:18:36.767492 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:36.767436    2551 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator_3508d0a07aa0fd49c57a7f8e3fd46334_0 in pod sandbox cece7048e2a160aa47f9b393b6bfca177a5beafee7e558b6416c41af5e024c52 from index: no such id: '75cb631ff271315d86bc225ab268ddb23e59b7d00093b57126e5248325fd8886'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334)\"" logger="UnhandledError"
Sep 10 13:18:36.767615 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:36.767596    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" podUID="3508d0a07aa0fd49c57a7f8e3fd46334"
Sep 10 13:18:36.778445 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:36.778385    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f82575a67e99  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334),Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:36.767542937 +0000 UTC m=+27.586581327,LastTimestamp:2026-09-10 13:18:36.767542937 +0000 UTC m=+27.586581327,Count:1,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:37.578506 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:37.578480    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:37.760863 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:37.760841    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/1.log"
Sep 10 13:18:38.572740 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:38.572715    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:39.571765 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:39.571732    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:39.638065 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:39.638032    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:18:40.253748 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:40.253712    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 13:18:40.571784 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:40.571715    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:41.235364 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:41.235337    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 13:18:41.238214 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:41.238185    2551 certificate_manager.go:596] "Failed while requesting a signed certificate from the control plane" err="cannot create certificate signing request: certificatesigningrequests.certificates.k8s.io is forbidden: User \"system:serviceaccount:openshift-machine-config-operator:node-bootstrapper\" cannot create resource \"certificatesigningrequests\" in API group \"certificates.k8s.io\" at the cluster scope" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError"
Sep 10 13:18:41.238278 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:41.238219    2551 certificate_manager.go:461] "Reached backoff limit, still unable to rotate certs" err="timed out waiting for the condition" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError"
Sep 10 13:18:41.571989 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:41.571933    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:41.953974 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:41.953909    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 13:18:42.572085 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:42.572058    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:43.257252 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:43.257222    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Sep 10 13:18:43.524003 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:43.523947    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:43.524969 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:43.524950    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:43.525030 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:43.524981    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:43.525030 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:43.524991    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:43.525030 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:43.525017    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:43.546267 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:43.546243    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:43.583460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:43.583440    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:44.053990 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:44.053963    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:44.054958 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:44.054938    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:44.055052 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:44.054975    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:44.055052 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:44.054987    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:44.572177 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:44.572156    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:45.578503 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:45.578476    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:46.573191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:46.573166    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:47.572291 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:47.572260    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:47.704602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:47.704562    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:47.705469 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:47.705451    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:47.705558 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:47.705481    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:47.705558 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:47.705491    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:47.705692 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:47.705673    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:47.715840 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:47.715757    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f822a8c94472\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f822a8c94472  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:24.740557938 +0000 UTC m=+15.559596317,LastTimestamp:2026-09-10 13:18:47.706586232 +0000 UTC m=+38.525624623,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:47.790337 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:47.790260    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f820022ae62c\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f820022ae62c  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:13.355226668 +0000 UTC m=+4.174265071,LastTimestamp:2026-09-10 13:18:47.77941922 +0000 UTC m=+38.598457598,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:47.798921 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:47.798861    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f820029c6928\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f820029c6928  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:13.362665768 +0000 UTC m=+4.181704166,LastTimestamp:2026-09-10 13:18:47.787712258 +0000 UTC m=+38.606750654,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:18:48.573251 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:48.573222    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:48.778195 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:48.778168    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/1.log"
Sep 10 13:18:48.778334 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:48.778221    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" event={"ID":"3508d0a07aa0fd49c57a7f8e3fd46334","Type":"ContainerStarted","Data":"e10c9b64ebd3a112ee7cfedae11ba355bed2c15bff08723295f41d20a689f128"}
Sep 10 13:18:48.778402 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:48.778347    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:48.780621 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:48.780606    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:48.780702 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:48.780635    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:48.780702 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:48.780645    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:48.780833 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:48.780821    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:49.572045 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:49.572017    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:49.638560 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:49.638523    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:18:50.269290 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:50.269262    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Sep 10 13:18:50.546766 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:50.546698    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:50.547698 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:50.547677    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:50.547826 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:50.547711    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:50.547826 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:50.547721    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:50.547826 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:50.547746    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:50.570625 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:50.570606    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:50.570731 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:50.570691    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:51.572497 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:51.572462    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:52.572772 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:52.572741    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:53.018684 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:53.018652    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 13:18:53.572284 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:53.572257    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:54.578617 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:54.578567    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:55.144335 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:55.144303    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 10 13:18:55.573231 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:55.573201    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:56.579735 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:56.579700    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:57.281026 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:57.280993    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Sep 10 13:18:57.571334 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:57.571265    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:57.572137 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:57.572122    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:57.572195 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:57.572154    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:57.572195 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:57.572165    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:57.572195 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:57.572191    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:57.573806 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:57.573787    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:57.591638 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:57.591611    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:58.574655 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:58.574625    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:59.579649 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:59.579619    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:18:59.639177 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:59.639154    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:18:59.795584 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:59.795556    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/2.log"
Sep 10 13:18:59.795932 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:59.795916    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/1.log"
Sep 10 13:18:59.795982 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:59.795953    2551 generic.go:356] "Generic (PLEG): container finished" podID="3508d0a07aa0fd49c57a7f8e3fd46334" containerID="e10c9b64ebd3a112ee7cfedae11ba355bed2c15bff08723295f41d20a689f128" exitCode=1
Sep 10 13:18:59.796015 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:59.795981    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" event={"ID":"3508d0a07aa0fd49c57a7f8e3fd46334","Type":"ContainerDied","Data":"e10c9b64ebd3a112ee7cfedae11ba355bed2c15bff08723295f41d20a689f128"}
Sep 10 13:18:59.796066 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:59.796056    2551 scope.go:117] "RemoveContainer" containerID="b56f860aaee41849315e936c4fbd75f107e29d60b21506ae24f32f2bfaa5f8a5"
Sep 10 13:18:59.796201 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:59.796184    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:18:59.797619 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:59.797595    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:18:59.797707 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:59.797634    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:18:59.797707 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:59.797645    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:18:59.797844 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:59.797832    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:18:59.797901 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:18:59.797878    2551 scope.go:117] "RemoveContainer" containerID="b56f860aaee41849315e936c4fbd75f107e29d60b21506ae24f32f2bfaa5f8a5"
Sep 10 13:18:59.805749 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:59.805724    2551 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator_3508d0a07aa0fd49c57a7f8e3fd46334_1 in pod sandbox cece7048e2a160aa47f9b393b6bfca177a5beafee7e558b6416c41af5e024c52 from index: no such id: 'b56f860aaee41849315e936c4fbd75f107e29d60b21506ae24f32f2bfaa5f8a5'" containerID="b56f860aaee41849315e936c4fbd75f107e29d60b21506ae24f32f2bfaa5f8a5"
Sep 10 13:18:59.805825 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:59.805764    2551 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator_3508d0a07aa0fd49c57a7f8e3fd46334_1 in pod sandbox cece7048e2a160aa47f9b393b6bfca177a5beafee7e558b6416c41af5e024c52 from index: no such id: 'b56f860aaee41849315e936c4fbd75f107e29d60b21506ae24f32f2bfaa5f8a5'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334)\"" logger="UnhandledError"
Sep 10 13:18:59.805903 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:59.805885    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" podUID="3508d0a07aa0fd49c57a7f8e3fd46334"
Sep 10 13:18:59.817879 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:18:59.817801    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f82575a67e99\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f82575a67e99  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334),Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:36.767542937 +0000 UTC m=+27.586581327,LastTimestamp:2026-09-10 13:18:59.805854653 +0000 UTC m=+50.624893044,Count:2,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:19:00.578495 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:00.578462    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:00.798521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:00.798494    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/2.log"
Sep 10 13:19:01.580108 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:01.580081    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:02.581327 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:02.581295    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:03.574971 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:03.574944    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:04.291983 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:04.291952    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Sep 10 13:19:04.574550 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:04.574487    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:04.591702 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:04.591672    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:19:04.592639 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:04.592623    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:19:04.592732 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:04.592656    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:19:04.592732 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:04.592671    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:19:04.592732 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:04.592703    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:04.631546 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:04.631520    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:05.575133 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:05.575102    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:06.585305 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:06.585276    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:07.585034 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:07.585000    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:08.618179 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:08.618150    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:09.573272 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:09.573240    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:09.640120 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:09.639973    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:10.582048 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:10.582019    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:11.377790 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:11.377753    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Sep 10 13:19:11.601884 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:11.601855    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:11.632214 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:11.632169    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:19:11.634017 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:11.634000    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:19:11.634115 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:11.634036    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:19:11.634115 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:11.634050    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:19:11.634115 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:11.634081    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:11.692397 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:11.692375    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:11.704615 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:11.704596    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:19:11.705382 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:11.705367    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:19:11.705464 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:11.705400    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:19:11.705464 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:11.705414    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:19:11.705650 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:11.705636    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:11.705811 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:11.705793    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" podUID="3508d0a07aa0fd49c57a7f8e3fd46334"
Sep 10 13:19:11.716977 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:11.716910    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f82575a67e99\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f82575a67e99  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334),Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:36.767542937 +0000 UTC m=+27.586581327,LastTimestamp:2026-09-10 13:19:11.705764102 +0000 UTC m=+62.524802496,Count:3,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:19:12.593787 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:12.593757    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:13.239564 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:13.239518    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 13:19:13.253521 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:13.253495    2551 certificate_manager.go:596] "Failed while requesting a signed certificate from the control plane" err="cannot create certificate signing request: certificatesigningrequests.certificates.k8s.io is forbidden: User \"system:serviceaccount:openshift-machine-config-operator:node-bootstrapper\" cannot create resource \"certificatesigningrequests\" in API group \"certificates.k8s.io\" at the cluster scope" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError"
Sep 10 13:19:13.581683 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:13.581607    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:14.594858 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:14.594832    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:15.593177 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:15.593149    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:16.572675 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:16.572649    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:17.113328 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:17.113297    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:19:17.114305 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:17.114289    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:19:17.114372 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:17.114320    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:19:17.114372 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:17.114331    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:19:17.574559 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:17.574537    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:17.819943 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:17.819921    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 13:19:18.388848 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:18.388817    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Sep 10 13:19:18.591121 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:18.591098    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:18.692921 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:18.692870    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:19:18.695672 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:18.695651    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:19:18.695764 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:18.695687    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:19:18.695764 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:18.695701    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:19:18.695764 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:18.695729    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:18.728023 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:18.727998    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:19.574985 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:19.574953    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:19.640374 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:19.640346    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:20.577222 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:20.577191    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:21.583813 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:21.583781    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:22.635224 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:22.635198    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:23.579960 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:23.579933    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:23.703792 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:23.703767    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:19:23.705307 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:23.705291    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:19:23.705382 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:23.705322    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:19:23.705382 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:23.705337    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:19:23.705616 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:23.705600    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:23.719082 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:23.718996    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f822a8c94472\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f822a8c94472  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:24.740557938 +0000 UTC m=+15.559596317,LastTimestamp:2026-09-10 13:19:23.70637345 +0000 UTC m=+74.525411841,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:19:23.789068 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:23.788998    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f820022ae62c\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f820022ae62c  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:13.355226668 +0000 UTC m=+4.174265071,LastTimestamp:2026-09-10 13:19:23.776786979 +0000 UTC m=+74.595825379,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:19:23.799493 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:23.799428    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f820029c6928\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f820029c6928  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:13.362665768 +0000 UTC m=+4.181704166,LastTimestamp:2026-09-10 13:19:23.784963333 +0000 UTC m=+74.604001723,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:19:23.833079 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:23.833036    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/2.log"
Sep 10 13:19:23.833149 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:23.833080    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" event={"ID":"3508d0a07aa0fd49c57a7f8e3fd46334","Type":"ContainerStarted","Data":"259f754dc9fe928fe2668d97125a97e69b2c0829c91de49b4b73ba252e507da0"}
Sep 10 13:19:23.833189 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:23.833183    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:19:23.833890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:23.833876    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:19:23.833941 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:23.833903    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:19:23.833941 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:23.833933    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:19:23.834139 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:23.834125    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:24.578192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:24.578165    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:25.404118 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:25.404085    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Sep 10 13:19:25.584930 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:25.584901    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:25.728741 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:25.728720    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:19:25.729570 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:25.729554    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:19:25.729570 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:25.729596    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:19:25.729711 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:25.729607    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:19:25.729711 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:25.729629    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:25.774262 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:25.774236    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:26.574034 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:26.574005    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:27.596255 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:27.596224    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:28.298852 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:28.298823    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 13:19:28.606912 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:28.606851    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:29.575393 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:29.575367    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:29.640940 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:29.640909    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:30.002736 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:30.002707    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 13:19:30.574653 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:30.574628    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:30.703746 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:30.703716    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:19:30.704642 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:30.704626    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:19:30.704706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:30.704659    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:19:30.704706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:30.704673    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:19:30.704918 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:30.704902    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:31.573566 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:31.573541    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:32.435759 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:32.435726    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Sep 10 13:19:32.592197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:32.592176    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:32.774460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:32.774433    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:19:32.775332 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:32.775314    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:19:32.775420 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:32.775348    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:19:32.775420 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:32.775361    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:19:32.775420 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:32.775383    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:32.813745 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:32.813726    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:33.580460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:33.580432    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:34.252941 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:34.252915    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 10 13:19:34.574852 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:34.574783    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:34.850613 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:34.850559    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/3.log"
Sep 10 13:19:34.850951 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:34.850938    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/2.log"
Sep 10 13:19:34.850993 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:34.850969    2551 generic.go:356] "Generic (PLEG): container finished" podID="3508d0a07aa0fd49c57a7f8e3fd46334" containerID="259f754dc9fe928fe2668d97125a97e69b2c0829c91de49b4b73ba252e507da0" exitCode=1
Sep 10 13:19:34.851027 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:34.850997    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" event={"ID":"3508d0a07aa0fd49c57a7f8e3fd46334","Type":"ContainerDied","Data":"259f754dc9fe928fe2668d97125a97e69b2c0829c91de49b4b73ba252e507da0"}
Sep 10 13:19:34.851059 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:34.851038    2551 scope.go:117] "RemoveContainer" containerID="e10c9b64ebd3a112ee7cfedae11ba355bed2c15bff08723295f41d20a689f128"
Sep 10 13:19:34.851126 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:34.851111    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:19:34.852113 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:34.851866    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:19:34.852113 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:34.851893    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:19:34.852113 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:34.851903    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:19:34.852221 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:34.852128    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:34.852221 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:34.852173    2551 scope.go:117] "RemoveContainer" containerID="e10c9b64ebd3a112ee7cfedae11ba355bed2c15bff08723295f41d20a689f128"
Sep 10 13:19:34.868622 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:34.868595    2551 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator_3508d0a07aa0fd49c57a7f8e3fd46334_2 in pod sandbox cece7048e2a160aa47f9b393b6bfca177a5beafee7e558b6416c41af5e024c52 from index: no such id: 'e10c9b64ebd3a112ee7cfedae11ba355bed2c15bff08723295f41d20a689f128'" containerID="e10c9b64ebd3a112ee7cfedae11ba355bed2c15bff08723295f41d20a689f128"
Sep 10 13:19:34.868703 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:34.868635    2551 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator_3508d0a07aa0fd49c57a7f8e3fd46334_2 in pod sandbox cece7048e2a160aa47f9b393b6bfca177a5beafee7e558b6416c41af5e024c52 from index: no such id: 'e10c9b64ebd3a112ee7cfedae11ba355bed2c15bff08723295f41d20a689f128'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334)\"" logger="UnhandledError"
Sep 10 13:19:34.868774 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:34.868758    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 40s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" podUID="3508d0a07aa0fd49c57a7f8e3fd46334"
Sep 10 13:19:34.878341 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:34.878272    2551 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f82575a67e99\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal.18d3f82575a67e99  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal,UID:3508d0a07aa0fd49c57a7f8e3fd46334,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334),Source:EventSource{Component:kubelet,Host:ip-10-0-135-117.ec2.internal,},FirstTimestamp:2026-09-10 13:18:36.767542937 +0000 UTC m=+27.586581327,LastTimestamp:2026-09-10 13:19:34.868727326 +0000 UTC m=+85.687765717,Count:4,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-117.ec2.internal,}"
Sep 10 13:19:35.578205 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:35.578176    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:35.854264 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:35.854199    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/3.log"
Sep 10 13:19:36.574192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:36.574167    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:37.575016 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:37.574990    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:38.575227 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:38.575197    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:39.450516 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:39.450482    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Sep 10 13:19:39.580737 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:39.580709    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:39.641299 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:39.641277    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:39.814565 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:39.814538    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:19:39.816240 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:39.816223    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:19:39.816315 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:39.816254    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:19:39.816315 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:39.816269    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:19:39.816315 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:39.816294    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:39.847891 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:39.847868    2551 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:40.575803 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:40.575774    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:41.575455 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:41.575428    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:42.578919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:42.578889    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:43.577979 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:43.577948    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:44.573761 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:44.573731    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:45.240269 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:45.240217    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 13:19:45.287882 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:45.287856    2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 13:19:45.576080 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:45.576016    2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k9lv4"
Sep 10 13:19:45.589132 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:45.589112    2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k9lv4"
Sep 10 13:19:45.605911 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:45.605893    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:19:46.500987 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:46.500952    2551 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 13:19:46.592324 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:46.592293    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 13:14:45 +0000 UTC" deadline="2028-04-22 17:03:08.357125124 +0000 UTC"
Sep 10 13:19:46.592324 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:46.592320    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14163h43m21.764808061s"
Sep 10 13:19:46.619146 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:46.619130    2551 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-117.ec2.internal" not found
Sep 10 13:19:46.647498 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:46.647483    2551 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-117.ec2.internal" not found
Sep 10 13:19:46.713895 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:46.713878    2551 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-117.ec2.internal" not found
Sep 10 13:19:46.848545 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:46.848496    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:19:46.849463 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:46.849433    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:19:46.849463 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:46.849467    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:19:46.849627 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:46.849480    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:19:46.849627 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:46.849505    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:46.870788 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:46.870767    2551 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:46.870870 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:46.870790    2551 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-117.ec2.internal\": node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:46.897586 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:46.897551    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:46.997668 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:46.997648    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:47.098395 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:47.098373    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:47.199059 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:47.199002    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:47.299540 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:47.299500    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:47.400070 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:47.400034    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:47.500920 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:47.500894    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:47.601486 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:47.601464    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:47.702291 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:47.702260    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:47.803368 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:47.803314    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:47.903761 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:47.903740    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:48.004296 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:48.004261    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:48.104961 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:48.104900    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:48.205376 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:48.205343    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:48.305884 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:48.305858    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:48.406925 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:48.406864    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:48.507528 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:48.507503    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:48.607625 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:48.607592    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:48.704451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:48.704384    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:19:48.705546 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:48.705526    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:19:48.705631 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:48.705556    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:19:48.705631 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:48.705566    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:19:48.705756 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:48.705744    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:19:48.705898 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:48.705883    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 40s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" podUID="3508d0a07aa0fd49c57a7f8e3fd46334"
Sep 10 13:19:48.707878 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:48.707862    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:48.808190 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:48.808163    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:48.908397 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:48.908374    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:49.009083 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:49.009053    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:49.109684 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:49.109650    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:49.209925 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:49.209892    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:49.310821 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:49.310765    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:49.411277 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:49.411256    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:49.512055 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:49.512023    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:49.612421 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:49.612365    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:49.641687 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:49.641665    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:49.712877 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:49.712856    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:49.813651 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:49.813632    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:49.913982 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:49.913930    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:50.014555 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:50.014525    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:50.115209 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:50.115186    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:50.216185 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:50.216134    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:50.316688 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:50.316667    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:50.417221 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:50.417183    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:50.517761 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:50.517743    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:50.618622 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:50.618599    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:50.719443 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:50.719420    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:50.820381 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:50.820305    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:50.920774 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:50.920745    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:51.021418 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:51.021386    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:51.122027 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:51.121965    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:51.222388 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:51.222360    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:51.322899 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:51.322870    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:51.423420 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:51.423370    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:51.524047 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:51.524020    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:51.624875 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:51.624840    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:51.725663 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:51.725637    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:51.825826 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:51.825806    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:51.926336 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:51.926302    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:52.026907 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:52.026856    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:52.126934 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:52.126900    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:52.227338 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:52.227315    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:52.327846 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:52.327796    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:52.390856 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:19:52.390830    2551 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:19:52.427861 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:52.427839    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:52.528498 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:52.528463    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:52.628737 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:52.628671    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:52.729661 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:52.729631    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:52.830184 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:52.830161    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:52.930686 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:52.930628    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:53.031309 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:53.031280    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:53.131958 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:53.131932    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:53.232352 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:53.232336    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:53.332897 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:53.332864    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:53.433408 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:53.433381    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:53.534528 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:53.534471    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:53.635557 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:53.635533    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:53.735775 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:53.735741    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:53.836659 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:53.836596    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:53.937043 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:53.937018    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:54.037551 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:54.037523    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:54.138112 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:54.138047    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:54.238395 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:54.238364    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:54.338715 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:54.338680    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:54.439552 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:54.439491    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:54.540257 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:54.540219    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:54.640601 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:54.640567    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:54.741688 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:54.741669    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:54.842293 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:54.842252    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:54.942703 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:54.942671    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:55.043383 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:55.043318    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:55.143956 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:55.143921    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:55.244215 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:55.244192    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:55.344703 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:55.344641    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:55.445252 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:55.445221    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:55.545783 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:55.545759    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:55.646799 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:55.646742    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:55.747544 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:55.747516    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:55.848328 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:55.848311    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:55.948653 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:55.948599    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:56.049283 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:56.049246    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:56.149883 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:56.149853    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:56.250409 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:56.250385    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:56.350856 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:56.350828    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:56.451806 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:56.451771    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:56.552785 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:56.552717    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:56.653842 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:56.653796    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:56.754635 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:56.754606    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:56.855183 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:56.855119    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:56.955480 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:56.955460    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:56.993894 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:56.993866    2551 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-117.ec2.internal\": node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:57.055862 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:57.055840    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:57.156427 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:57.156369    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:57.256794 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:57.256774    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:57.357331 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:57.357304    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:57.457832 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:57.457773    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:57.558579 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:57.558548    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:57.659503 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:57.659472    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:57.760329 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:57.760313    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:57.860760 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:57.860736    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:57.961449 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:57.961417    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:58.062154 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:58.062103    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:58.162581 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:58.162553    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:58.263084 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:58.263060    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:58.363695 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:58.363643    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:58.464191 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:58.464164    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:58.564896 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:58.564872    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:58.665985 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:58.665924    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:58.766809 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:58.766771    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:58.867293 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:58.867273    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:58.967759 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:58.967711    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:59.068473 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:59.068445    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:59.168933 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:59.168909    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:59.269442 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:59.269425    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:59.369954 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:59.369926    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:59.470413 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:59.470391    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:59.570817 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:59.570772    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:59.642637 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:59.642609    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:59.671498 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:59.671480    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:59.772120 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:59.772097    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:59.872584 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:59.872509    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:19:59.972918 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:19:59.972885    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:00.073677 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:00.073640    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:00.174241 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:00.174176    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:00.274672 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:00.274639    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:00.375199 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:00.375171    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:00.476076 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:00.476053    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:00.576939 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:00.576916    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:00.677652 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:00.677622    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:00.704361 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:00.704339    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:20:00.706687 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:00.706671    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:20:00.706785 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:00.706708    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:20:00.706785 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:00.706723    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:20:00.706987 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:00.706971    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-117.ec2.internal\" not found" node="ip-10-0-135-117.ec2.internal"
Sep 10 13:20:00.707158 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:00.707138    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 40s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" podUID="3508d0a07aa0fd49c57a7f8e3fd46334"
Sep 10 13:20:00.778476 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:00.778411    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:00.878947 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:00.878914    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:00.979293 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:00.979271    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:01.080108 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:01.080057    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:01.180989 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:01.180964    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:01.281443 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:01.281415    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:01.381992 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:01.381935    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:01.482386 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:01.482355    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:01.582434 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:01.582407    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:01.683165 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:01.683103    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:01.783887 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:01.783860    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:01.884411 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:01.884389    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:01.984787 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:01.984760    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:02.085528 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:02.085508    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:02.186071 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:02.186037    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:02.286488 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:02.286432    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:02.387350 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:02.387320    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:02.487981 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:02.487947    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:02.588621 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:02.588551    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:02.689021 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:02.688983    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:02.789706 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:02.789675    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:02.890810 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:02.890737    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:02.991161 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:02.991135    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:03.091960 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:03.091930    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:03.192397 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:03.192338    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:03.292855 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:03.292822    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:03.393325 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:03.393296    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:03.493760 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:03.493740    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:03.594567 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:03.594543    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:03.695329 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:03.695305    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:03.796155 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:03.796098    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:03.896500 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:03.896479    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:03.996976 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:03.996949    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:04.097996 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:04.097941    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:04.198383 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:04.198355    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:04.299008 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:04.298986    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:04.399437 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:04.399393    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:04.500077 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:04.500051    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:04.600889 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:04.600860    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:04.701276 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:04.701214    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:04.801987 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:04.801958    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:04.902379 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:04.902355    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:05.003324 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:05.003305    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:05.104049 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:05.104027    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:05.204450 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:05.204426    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:05.305009 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:05.304956    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:05.405386 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:05.405362    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:05.505960 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:05.505931    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:05.606695 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:05.606640    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:05.707523 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:05.707499    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:05.808167 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:05.808139    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:05.909089 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:05.909041    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:06.009614 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:06.009589    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:06.110558 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:06.110537    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:06.211078 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:06.211029    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:06.311450 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:06.311431    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:06.411980 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:06.411955    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:06.512541 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:06.512522    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:06.613308 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:06.613284    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:06.714207 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:06.714181    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:06.814678 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:06.814624    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:06.915058 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:06.915040    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:07.015607 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:07.015593    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:07.113019 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:07.112967    2551 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-117.ec2.internal\": node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:07.118303 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:07.118288    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:07.218597 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:07.218565    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:07.319115 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:07.319090    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:07.420157 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:07.420113    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:07.520711 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:07.520678    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:07.621657 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:07.621626    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:07.722446 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:07.722400    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:07.822918 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:07.822891    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:07.923323 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:07.923298    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:08.024327 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:08.024303    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:08.124681 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:08.124647    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:08.225616 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:08.225583    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:08.326121 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:08.326079    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:08.426513 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:08.426487    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:08.527177 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:08.527152    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:08.627973 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:08.627922    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:08.728735 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:08.728711    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:08.829246 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:08.829213    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:08.929604 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:08.929542    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:09.030189 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:09.030163    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:09.130901 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:09.130878    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:09.231397 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:09.231380    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:09.331897 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:09.331872    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:09.432389 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:09.432368    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:09.533399 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:09.533353    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:09.633793 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:09.633774    2551 kubelet_node_status.go:448] "Node not becoming ready in time after startup"
Sep 10 13:20:09.643143 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:09.643127    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-117.ec2.internal\" not found"
Sep 10 13:20:09.660772 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:09.660741    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:20:12.398558 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.398528    2551 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:20:12.470950 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.470927    2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal"
Sep 10 13:20:12.510929 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.510912    2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 13:20:12.511746 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.511734    2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-117.ec2.internal"
Sep 10 13:20:12.520928 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.520914    2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 13:20:12.622633 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.622606    2551 apiserver.go:52] "Watching apiserver"
Sep 10 13:20:12.627971 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.627956    2551 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 13:20:12.628149 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.628133    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-t85bm","kube-system/kube-apiserver-proxy-ip-10-0-135-117.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal"]
Sep 10 13:20:12.630467 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.630455    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t85bm"
Sep 10 13:20:12.630612 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:12.630586    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 40s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" podUID="3508d0a07aa0fd49c57a7f8e3fd46334"
Sep 10 13:20:12.638057 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.638024    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 13:20:12.638427 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.638415    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 13:20:12.644325 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.644311    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 13:20:12.672251 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.672216    2551 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 13:20:12.674348 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.674333    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60d72c3e-783a-4a4c-918c-02070def1e7b-agent-certs\") pod \"konnectivity-agent-t85bm\" (UID: \"60d72c3e-783a-4a4c-918c-02070def1e7b\") " pod="kube-system/konnectivity-agent-t85bm"
Sep 10 13:20:12.674399 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.674356    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60d72c3e-783a-4a4c-918c-02070def1e7b-konnectivity-ca\") pod \"konnectivity-agent-t85bm\" (UID: \"60d72c3e-783a-4a4c-918c-02070def1e7b\") " pod="kube-system/konnectivity-agent-t85bm"
Sep 10 13:20:12.677367 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.677349    2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 13:20:12.685973 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.685938    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-117.ec2.internal" podStartSLOduration=0.685928173 podStartE2EDuration="685.928173ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:12.685319453 +0000 UTC m=+123.504357852" watchObservedRunningTime="2026-09-10 13:20:12.685928173 +0000 UTC m=+123.504966569"
Sep 10 13:20:12.737620 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.737598    2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4bkcv"
Sep 10 13:20:12.774933 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.774916    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60d72c3e-783a-4a4c-918c-02070def1e7b-agent-certs\") pod \"konnectivity-agent-t85bm\" (UID: \"60d72c3e-783a-4a4c-918c-02070def1e7b\") " pod="kube-system/konnectivity-agent-t85bm"
Sep 10 13:20:12.775011 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.774945    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60d72c3e-783a-4a4c-918c-02070def1e7b-konnectivity-ca\") pod \"konnectivity-agent-t85bm\" (UID: \"60d72c3e-783a-4a4c-918c-02070def1e7b\") " pod="kube-system/konnectivity-agent-t85bm"
Sep 10 13:20:12.775396 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.775376    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60d72c3e-783a-4a4c-918c-02070def1e7b-konnectivity-ca\") pod \"konnectivity-agent-t85bm\" (UID: \"60d72c3e-783a-4a4c-918c-02070def1e7b\") " pod="kube-system/konnectivity-agent-t85bm"
Sep 10 13:20:12.775499 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.775393    2551 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 13:20:12.778860 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.778845    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60d72c3e-783a-4a4c-918c-02070def1e7b-agent-certs\") pod \"konnectivity-agent-t85bm\" (UID: \"60d72c3e-783a-4a4c-918c-02070def1e7b\") " pod="kube-system/konnectivity-agent-t85bm"
Sep 10 13:20:12.938027 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:12.937973    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t85bm"
Sep 10 13:20:12.946363 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:20:12.946342    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60d72c3e_783a_4a4c_918c_02070def1e7b.slice/crio-c1da7b0b5e26c7783799abdc7c7ab635078177b9837d9c7f44d29f194bd72fdf WatchSource:0}: Error finding container c1da7b0b5e26c7783799abdc7c7ab635078177b9837d9c7f44d29f194bd72fdf: Status 404 returned error can't find the container with id c1da7b0b5e26c7783799abdc7c7ab635078177b9837d9c7f44d29f194bd72fdf
Sep 10 13:20:13.908268 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:13.908230    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t85bm" event={"ID":"60d72c3e-783a-4a4c-918c-02070def1e7b","Type":"ContainerStarted","Data":"c1da7b0b5e26c7783799abdc7c7ab635078177b9837d9c7f44d29f194bd72fdf"}
Sep 10 13:20:14.220656 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:14.220564    2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:20:14.661958 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:14.661925    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:20:14.911419 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:14.911390    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t85bm" event={"ID":"60d72c3e-783a-4a4c-918c-02070def1e7b","Type":"ContainerStarted","Data":"da48388c720d222f06d59da5d332f4fe62d714bb4b21cf1519b292c93db557e2"}
Sep 10 13:20:14.930924 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:14.930836    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-t85bm" podStartSLOduration=27.488469224 podStartE2EDuration="28.930823821s" totalImagesPullingTime="1.442354597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:19:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:14.930803609 +0000 UTC m=+125.749842009" watchObservedRunningTime="2026-09-10 13:20:14.930823821 +0000 UTC m=+125.749862221"
Sep 10 13:20:17.938649 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:17.938615    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-t85bm"
Sep 10 13:20:17.939168 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:17.939147    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-t85bm"
Sep 10 13:20:18.917472 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:18.917440    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-t85bm"
Sep 10 13:20:18.917889 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:18.917874    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-t85bm"
Sep 10 13:20:19.662433 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:19.662401    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:20:24.663426 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:24.663390    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:20:25.914183 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:25.914161    2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:20:25.927936 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:25.927908    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/3.log"
Sep 10 13:20:25.928036 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:25.927960    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" event={"ID":"3508d0a07aa0fd49c57a7f8e3fd46334","Type":"ContainerStarted","Data":"ac174718848282054933483bfb3a82fb876466bea9556b7e8b789bee597ecfe4"}
Sep 10 13:20:26.116288 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.116262    2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4bkcv"
Sep 10 13:20:26.633138 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.633116    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-ln5qf"]
Sep 10 13:20:26.635515 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.635500    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.644592 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.644561    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-system-cni-dir\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.644680 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.644599    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-cnibin\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.644680 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.644616    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.644680 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.644631    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.644680 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.644645    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfg7m\" (UniqueName: \"kubernetes.io/projected/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-kube-api-access-jfg7m\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.644680 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.644661    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-cni-binary-copy\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.644680 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.644677    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.742823 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.742801    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 13:20:26.746052 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.746027    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-cnibin\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.746162 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.746069    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-cnibin\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.746162 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.746074    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.746162 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.746113    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.746162 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.746130    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfg7m\" (UniqueName: \"kubernetes.io/projected/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-kube-api-access-jfg7m\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.746162 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.746154    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-cni-binary-copy\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.746352 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.746170    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.746352 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.746186    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-system-cni-dir\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.746352 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.746229    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-system-cni-dir\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.746601 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.746567    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.746835 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.746818    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.776674 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.776659    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 13:20:26.780742 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.780723    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 13:20:26.783732 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.783714    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 13:20:26.787131 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.787112    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.821971 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.821950    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 13:20:26.822078 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:26.821954    2551 projected.go:291] Couldn't get configMap openshift-multus/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Sep 10 13:20:26.822078 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:26.822058    2551 projected.go:291] Couldn't get configMap openshift-multus/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found
Sep 10 13:20:26.822078 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:26.822071    2551 projected.go:196] Error preparing data for projected volume kube-api-access-jfg7m for pod openshift-multus/multus-additional-cni-plugins-ln5qf: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Sep 10 13:20:26.822215 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:26.822124    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-kube-api-access-jfg7m podName:d37fb884-c078-4fd0-baf5-cd0ee01f2a7c nodeName:}" failed. No retries permitted until 2026-09-10 13:20:27.322110296 +0000 UTC m=+138.141148686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jfg7m" (UniqueName: "kubernetes.io/projected/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-kube-api-access-jfg7m") pod "multus-additional-cni-plugins-ln5qf" (UID: "d37fb884-c078-4fd0-baf5-cd0ee01f2a7c") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Sep 10 13:20:26.822215 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.822145    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7ztgj\""
Sep 10 13:20:26.827725 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.827708    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-cni-binary-copy\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:26.876234 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.876214    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-mtb95"]
Sep 10 13:20:26.878605 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.878591    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.889188 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.889169    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 13:20:26.931660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.931642    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_3508d0a07aa0fd49c57a7f8e3fd46334/setup/3.log"
Sep 10 13:20:26.931904 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.931677    2551 generic.go:356] "Generic (PLEG): container finished" podID="3508d0a07aa0fd49c57a7f8e3fd46334" containerID="ac174718848282054933483bfb3a82fb876466bea9556b7e8b789bee597ecfe4" exitCode=0
Sep 10 13:20:26.931904 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.931706    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" event={"ID":"3508d0a07aa0fd49c57a7f8e3fd46334","Type":"ContainerDied","Data":"ac174718848282054933483bfb3a82fb876466bea9556b7e8b789bee597ecfe4"}
Sep 10 13:20:26.931904 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.931739    2551 scope.go:117] "RemoveContainer" containerID="259f754dc9fe928fe2668d97125a97e69b2c0829c91de49b4b73ba252e507da0"
Sep 10 13:20:26.932017 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.931930    2551 scope.go:117] "RemoveContainer" containerID="259f754dc9fe928fe2668d97125a97e69b2c0829c91de49b4b73ba252e507da0"
Sep 10 13:20:26.940975 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:26.940953    2551 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator_3508d0a07aa0fd49c57a7f8e3fd46334_3 in pod sandbox cece7048e2a160aa47f9b393b6bfca177a5beafee7e558b6416c41af5e024c52 from index: no such id: '259f754dc9fe928fe2668d97125a97e69b2c0829c91de49b4b73ba252e507da0'" containerID="259f754dc9fe928fe2668d97125a97e69b2c0829c91de49b4b73ba252e507da0"
Sep 10 13:20:26.941048 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:26.940991    2551 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator_3508d0a07aa0fd49c57a7f8e3fd46334_3 in pod sandbox cece7048e2a160aa47f9b393b6bfca177a5beafee7e558b6416c41af5e024c52 from index: no such id: '259f754dc9fe928fe2668d97125a97e69b2c0829c91de49b4b73ba252e507da0'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal_openshift-machine-config-operator(3508d0a07aa0fd49c57a7f8e3fd46334)\"" logger="UnhandledError"
Sep 10 13:20:26.947999 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.947979    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-var-lib-cni-bin\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948002    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-var-lib-kubelet\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948016    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-multus-conf-dir\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948031    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5ec389cc-200a-4541-a957-8f419ee31f55-cni-binary-copy\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948045    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-run-k8s-cni-cncf-io\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948062    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-hostroot\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948098    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r47j\" (UniqueName: \"kubernetes.io/projected/5ec389cc-200a-4541-a957-8f419ee31f55-kube-api-access-5r47j\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948145    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-multus-socket-dir-parent\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948168    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-run-multus-certs\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948184    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-system-cni-dir\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948200    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-multus-cni-dir\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948222    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-run-netns\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948240    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-var-lib-cni-multus\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948254    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5ec389cc-200a-4541-a957-8f419ee31f55-multus-daemon-config\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948268    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-etc-kubernetes\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:26.948313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:26.948300    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-cnibin\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049482 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049458    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-system-cni-dir\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049615 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049485    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-multus-cni-dir\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049615 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049503    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-run-netns\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049615 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049519    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-var-lib-cni-multus\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049615 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049535    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5ec389cc-200a-4541-a957-8f419ee31f55-multus-daemon-config\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049615 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049557    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-etc-kubernetes\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049615 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049599    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-run-netns\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049615 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049613    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-cnibin\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049611    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-multus-cni-dir\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049650    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-var-lib-cni-bin\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049671    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-etc-kubernetes\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049669    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-system-cni-dir\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049678    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-var-lib-kubelet\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049611    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-var-lib-cni-multus\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049710    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-var-lib-kubelet\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049715    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-var-lib-cni-bin\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049717    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-multus-conf-dir\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049712    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-cnibin\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049753    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-multus-conf-dir\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049792    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5ec389cc-200a-4541-a957-8f419ee31f55-cni-binary-copy\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049808    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-run-k8s-cni-cncf-io\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049821    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-hostroot\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049835    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r47j\" (UniqueName: \"kubernetes.io/projected/5ec389cc-200a-4541-a957-8f419ee31f55-kube-api-access-5r47j\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049851    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-multus-socket-dir-parent\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049867    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-run-multus-certs\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.049925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049890    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-hostroot\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.050602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049891    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-run-k8s-cni-cncf-io\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.050602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049912    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-host-run-multus-certs\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.050602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.049936    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5ec389cc-200a-4541-a957-8f419ee31f55-multus-socket-dir-parent\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.050602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.050145    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5ec389cc-200a-4541-a957-8f419ee31f55-multus-daemon-config\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.050602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.050274    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5ec389cc-200a-4541-a957-8f419ee31f55-cni-binary-copy\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.078881 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.078862    2551 projected.go:291] Couldn't get configMap openshift-multus/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.078938 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.078886    2551 projected.go:196] Error preparing data for projected volume kube-api-access-5r47j for pod openshift-multus/multus-mtb95: configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.078938 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.078933    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ec389cc-200a-4541-a957-8f419ee31f55-kube-api-access-5r47j podName:5ec389cc-200a-4541-a957-8f419ee31f55 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:27.578914589 +0000 UTC m=+138.397952969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5r47j" (UniqueName: "kubernetes.io/projected/5ec389cc-200a-4541-a957-8f419ee31f55-kube-api-access-5r47j") pod "multus-mtb95" (UID: "5ec389cc-200a-4541-a957-8f419ee31f55") : configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.119612 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.119559    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 13:15:26 +0000 UTC" deadline="2028-04-29 19:48:58.070248639 +0000 UTC"
Sep 10 13:20:27.119612 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.119611    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14334h28m30.950641921s"
Sep 10 13:20:27.191695 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.191642    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-xxmwh"]
Sep 10 13:20:27.194333 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.194317    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-82cgh"]
Sep 10 13:20:27.194430 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.194417    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:27.194518 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.194497    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:27.196583 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.196558    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:27.196631 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.196618    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:27.250584 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.250552    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrm9s\" (UniqueName: \"kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s\") pod \"network-check-target-82cgh\" (UID: \"4c893996-bc6b-46f0-ab3f-b98dcb115908\") " pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:27.250658 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.250598    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:27.250658 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.250615    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt7jx\" (UniqueName: \"kubernetes.io/projected/8bc0fa69-2ca1-4cd6-b004-88863086d37d-kube-api-access-vt7jx\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:27.313179 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.313158    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-k4k7c"]
Sep 10 13:20:27.315505 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.315492    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k4k7c"
Sep 10 13:20:27.321956 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.321937    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 13:20:27.322067 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.321974    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 13:20:27.322067 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.321983    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 13:20:27.322215 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.322202    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-czbtf\""
Sep 10 13:20:27.351603 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.351560    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrm9s\" (UniqueName: \"kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s\") pod \"network-check-target-82cgh\" (UID: \"4c893996-bc6b-46f0-ab3f-b98dcb115908\") " pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:27.351675 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.351611    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfg7m\" (UniqueName: \"kubernetes.io/projected/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-kube-api-access-jfg7m\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:27.351675 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.351629    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:27.351675 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.351645    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt7jx\" (UniqueName: \"kubernetes.io/projected/8bc0fa69-2ca1-4cd6-b004-88863086d37d-kube-api-access-vt7jx\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:27.351829 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.351713    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:20:27.351829 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.351739    2551 projected.go:291] Couldn't get configMap openshift-multus/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.351829 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.351765    2551 projected.go:196] Error preparing data for projected volume kube-api-access-jfg7m for pod openshift-multus/multus-additional-cni-plugins-ln5qf: configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.351829 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.351761    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkjzt\" (UniqueName: \"kubernetes.io/projected/ad5b6dca-ed23-4c99-8b04-68a1afae4e1f-kube-api-access-rkjzt\") pod \"iptables-alerter-k4k7c\" (UID: \"ad5b6dca-ed23-4c99-8b04-68a1afae4e1f\") " pod="openshift-network-operator/iptables-alerter-k4k7c"
Sep 10 13:20:27.351829 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.351768    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs podName:8bc0fa69-2ca1-4cd6-b004-88863086d37d nodeName:}" failed. No retries permitted until 2026-09-10 13:20:27.851751949 +0000 UTC m=+138.670790332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs") pod "network-metrics-daemon-xxmwh" (UID: "8bc0fa69-2ca1-4cd6-b004-88863086d37d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:20:27.351829 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.351827    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-kube-api-access-jfg7m podName:d37fb884-c078-4fd0-baf5-cd0ee01f2a7c nodeName:}" failed. No retries permitted until 2026-09-10 13:20:28.351809739 +0000 UTC m=+139.170848132 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jfg7m" (UniqueName: "kubernetes.io/projected/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-kube-api-access-jfg7m") pod "multus-additional-cni-plugins-ln5qf" (UID: "d37fb884-c078-4fd0-baf5-cd0ee01f2a7c") : configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.352029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.351853    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ad5b6dca-ed23-4c99-8b04-68a1afae4e1f-iptables-alerter-script\") pod \"iptables-alerter-k4k7c\" (UID: \"ad5b6dca-ed23-4c99-8b04-68a1afae4e1f\") " pod="openshift-network-operator/iptables-alerter-k4k7c"
Sep 10 13:20:27.352029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.351869    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad5b6dca-ed23-4c99-8b04-68a1afae4e1f-host-slash\") pod \"iptables-alerter-k4k7c\" (UID: \"ad5b6dca-ed23-4c99-8b04-68a1afae4e1f\") " pod="openshift-network-operator/iptables-alerter-k4k7c"
Sep 10 13:20:27.394147 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.394131    2551 projected.go:291] Couldn't get configMap openshift-multus/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.394199 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.394155    2551 projected.go:196] Error preparing data for projected volume kube-api-access-vt7jx for pod openshift-multus/network-metrics-daemon-xxmwh: configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.394199 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.394197    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bc0fa69-2ca1-4cd6-b004-88863086d37d-kube-api-access-vt7jx podName:8bc0fa69-2ca1-4cd6-b004-88863086d37d nodeName:}" failed. No retries permitted until 2026-09-10 13:20:27.894183798 +0000 UTC m=+138.713222185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vt7jx" (UniqueName: "kubernetes.io/projected/8bc0fa69-2ca1-4cd6-b004-88863086d37d-kube-api-access-vt7jx") pod "network-metrics-daemon-xxmwh" (UID: "8bc0fa69-2ca1-4cd6-b004-88863086d37d") : configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.410797 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.410780    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-fnfwb"]
Sep 10 13:20:27.411173 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.411161    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:20:27.411215 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.411178    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:20:27.411215 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.411187    2551 projected.go:196] Error preparing data for projected volume kube-api-access-xrm9s for pod openshift-network-diagnostics/network-check-target-82cgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:20:27.411275 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.411224    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s podName:4c893996-bc6b-46f0-ab3f-b98dcb115908 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:27.911211486 +0000 UTC m=+138.730249866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xrm9s" (UniqueName: "kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s") pod "network-check-target-82cgh" (UID: "4c893996-bc6b-46f0-ab3f-b98dcb115908") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:20:27.413244 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.413231    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:27.413293 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.413279    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:27.452740 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.452691    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ad5b6dca-ed23-4c99-8b04-68a1afae4e1f-iptables-alerter-script\") pod \"iptables-alerter-k4k7c\" (UID: \"ad5b6dca-ed23-4c99-8b04-68a1afae4e1f\") " pod="openshift-network-operator/iptables-alerter-k4k7c"
Sep 10 13:20:27.452740 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.452718    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad5b6dca-ed23-4c99-8b04-68a1afae4e1f-host-slash\") pod \"iptables-alerter-k4k7c\" (UID: \"ad5b6dca-ed23-4c99-8b04-68a1afae4e1f\") " pod="openshift-network-operator/iptables-alerter-k4k7c"
Sep 10 13:20:27.452740 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.452735    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/68de88a2-c141-4c07-9bfd-9ad2aa25b835-kubelet-config\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:27.452890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.452759    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/68de88a2-c141-4c07-9bfd-9ad2aa25b835-dbus\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:27.452890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.452806    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad5b6dca-ed23-4c99-8b04-68a1afae4e1f-host-slash\") pod \"iptables-alerter-k4k7c\" (UID: \"ad5b6dca-ed23-4c99-8b04-68a1afae4e1f\") " pod="openshift-network-operator/iptables-alerter-k4k7c"
Sep 10 13:20:27.452890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.452844    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:27.452890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.452888    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkjzt\" (UniqueName: \"kubernetes.io/projected/ad5b6dca-ed23-4c99-8b04-68a1afae4e1f-kube-api-access-rkjzt\") pod \"iptables-alerter-k4k7c\" (UID: \"ad5b6dca-ed23-4c99-8b04-68a1afae4e1f\") " pod="openshift-network-operator/iptables-alerter-k4k7c"
Sep 10 13:20:27.453172 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.453156    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ad5b6dca-ed23-4c99-8b04-68a1afae4e1f-iptables-alerter-script\") pod \"iptables-alerter-k4k7c\" (UID: \"ad5b6dca-ed23-4c99-8b04-68a1afae4e1f\") " pod="openshift-network-operator/iptables-alerter-k4k7c"
Sep 10 13:20:27.489454 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.489429    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkjzt\" (UniqueName: \"kubernetes.io/projected/ad5b6dca-ed23-4c99-8b04-68a1afae4e1f-kube-api-access-rkjzt\") pod \"iptables-alerter-k4k7c\" (UID: \"ad5b6dca-ed23-4c99-8b04-68a1afae4e1f\") " pod="openshift-network-operator/iptables-alerter-k4k7c"
Sep 10 13:20:27.509897 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.509878    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-7h6v4"]
Sep 10 13:20:27.512649 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.512635    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.520937 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.520912    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 13:20:27.521058 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.521042    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 13:20:27.521109 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.521063    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 13:20:27.521357 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.521343    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vx2vm\""
Sep 10 13:20:27.521432 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.521349    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 13:20:27.521777 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.521763    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 13:20:27.522300 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.522286    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 13:20:27.530910 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.529733    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"]
Sep 10 13:20:27.533556 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.533540    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.535905 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.535883    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 13:20:27.535996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.535917    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 13:20:27.536503 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.536490    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mzt68\""
Sep 10 13:20:27.537115 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.537098    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 13:20:27.553191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553163    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-slash\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.553285 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553202    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-openvswitch\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.553285 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553233    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-socket-dir\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.553285 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553250    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-registration-dir\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.553285 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553272    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-sys-fs\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.553521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553294    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-run-netns\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.553521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553319    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-etc-openvswitch\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.553521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553356    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovnkube-config\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.553521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553378    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovnkube-script-lib\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.553521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553403    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-var-lib-openvswitch\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.553521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553425    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.553521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553464    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.553521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553488    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-device-dir\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.553521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553523    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-ovn\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553549    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553564    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovn-node-metrics-cert\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553612    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-etc-selinux\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553627    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-systemd-units\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553641    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-cni-bin\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553660    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-kubelet\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.553668    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553693    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.553751    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret podName:68de88a2-c141-4c07-9bfd-9ad2aa25b835 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:28.053732508 +0000 UTC m=+138.872770900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret") pod "global-pull-secret-syncer-fnfwb" (UID: "68de88a2-c141-4c07-9bfd-9ad2aa25b835") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553851    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-cni-netd\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553880    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnmsg\" (UniqueName: \"kubernetes.io/projected/28f104e3-2da9-4ad6-bb98-361fd93879cd-kube-api-access-cnmsg\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553922    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxkqf\" (UniqueName: \"kubernetes.io/projected/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-kube-api-access-mxkqf\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553946    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-log-socket\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553965    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-node-log\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.553983    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/68de88a2-c141-4c07-9bfd-9ad2aa25b835-kubelet-config\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:27.554041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.554002    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/68de88a2-c141-4c07-9bfd-9ad2aa25b835-dbus\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:27.554564 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.554026    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-systemd\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.554564 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.554053    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/68de88a2-c141-4c07-9bfd-9ad2aa25b835-kubelet-config\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:27.554564 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.554089    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-env-overrides\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.554564 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.554132    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/68de88a2-c141-4c07-9bfd-9ad2aa25b835-dbus\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:27.617972 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.617950    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-l9cw9"]
Sep 10 13:20:27.620382 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.620369    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l9cw9"
Sep 10 13:20:27.623141 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.623126    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k4k7c"
Sep 10 13:20:27.631730 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:20:27.631711    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad5b6dca_ed23_4c99_8b04_68a1afae4e1f.slice/crio-5ce617793caff99d63ea0e6757005947e64aa387e0aea6d8ddc2ae85d9136e00 WatchSource:0}: Error finding container 5ce617793caff99d63ea0e6757005947e64aa387e0aea6d8ddc2ae85d9136e00: Status 404 returned error can't find the container with id 5ce617793caff99d63ea0e6757005947e64aa387e0aea6d8ddc2ae85d9136e00
Sep 10 13:20:27.636863 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.636840    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 13:20:27.636863 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.636853    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rg9tv\""
Sep 10 13:20:27.637015 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.636838    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 13:20:27.637015 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.636898    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 13:20:27.655272 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655248    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-systemd\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.655366 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655280    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4a5a18c4-1ee9-4a34-b304-49b3f6f26674-serviceca\") pod \"node-ca-l9cw9\" (UID: \"4a5a18c4-1ee9-4a34-b304-49b3f6f26674\") " pod="openshift-image-registry/node-ca-l9cw9"
Sep 10 13:20:27.655366 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655306    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-systemd\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.655366 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655309    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r47j\" (UniqueName: \"kubernetes.io/projected/5ec389cc-200a-4541-a957-8f419ee31f55-kube-api-access-5r47j\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:27.655366 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655346    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-env-overrides\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.655600 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655389    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr78x\" (UniqueName: \"kubernetes.io/projected/4a5a18c4-1ee9-4a34-b304-49b3f6f26674-kube-api-access-sr78x\") pod \"node-ca-l9cw9\" (UID: \"4a5a18c4-1ee9-4a34-b304-49b3f6f26674\") " pod="openshift-image-registry/node-ca-l9cw9"
Sep 10 13:20:27.655600 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.655397    2551 projected.go:291] Couldn't get configMap openshift-multus/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.655600 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655413    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-slash\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.655600 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.655418    2551 projected.go:196] Error preparing data for projected volume kube-api-access-5r47j for pod openshift-multus/multus-mtb95: configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.655600 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655431    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-openvswitch\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.655600 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655459    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-socket-dir\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.655600 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.655471    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ec389cc-200a-4541-a957-8f419ee31f55-kube-api-access-5r47j podName:5ec389cc-200a-4541-a957-8f419ee31f55 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:28.655453944 +0000 UTC m=+139.474492336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5r47j" (UniqueName: "kubernetes.io/projected/5ec389cc-200a-4541-a957-8f419ee31f55-kube-api-access-5r47j") pod "multus-mtb95" (UID: "5ec389cc-200a-4541-a957-8f419ee31f55") : configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.655600 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655475    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-slash\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.655600 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655500    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-registration-dir\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.655600 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655525    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-sys-fs\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.655600 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655549    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-run-netns\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.655600 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655592    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-socket-dir\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.655600 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655596    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-etc-openvswitch\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655620    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-run-netns\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655629    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-sys-fs\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655630    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovnkube-config\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655647    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-registration-dir\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655678    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovnkube-script-lib\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655696    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-etc-openvswitch\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655705    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a5a18c4-1ee9-4a34-b304-49b3f6f26674-host\") pod \"node-ca-l9cw9\" (UID: \"4a5a18c4-1ee9-4a34-b304-49b3f6f26674\") " pod="openshift-image-registry/node-ca-l9cw9"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655731    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-var-lib-openvswitch\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655734    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-openvswitch\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655787    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-var-lib-openvswitch\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655828    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655854    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655869    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-device-dir\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655899    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-ovn\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655927    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655947    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovn-node-metrics-cert\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655903    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655965    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-device-dir\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655974    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-etc-selinux\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655981    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-ovn\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.655991    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-systemd-units\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656006    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-cni-bin\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656024    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-kubelet\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656042    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656060    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-systemd-units\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656065    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-etc-selinux\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656070    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-cni-bin\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656089    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656097    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-kubelet\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656134    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-cni-netd\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656163    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-cni-netd\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656163    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnmsg\" (UniqueName: \"kubernetes.io/projected/28f104e3-2da9-4ad6-bb98-361fd93879cd-kube-api-access-cnmsg\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656198    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxkqf\" (UniqueName: \"kubernetes.io/projected/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-kube-api-access-mxkqf\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.656919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656213    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-log-socket\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.657468 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656227    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-node-log\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.657468 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656271    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-node-log\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.657468 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656275    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-log-socket\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.657468 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656600    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-env-overrides\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.657468 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656625    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovnkube-config\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.657468 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.656713    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovnkube-script-lib\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.658309 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.658293    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovn-node-metrics-cert\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.688870 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.688851    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxkqf\" (UniqueName: \"kubernetes.io/projected/e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd-kube-api-access-mxkqf\") pod \"aws-ebs-csi-driver-node-9m8l6\" (UID: \"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.701441 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.701425    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnmsg\" (UniqueName: \"kubernetes.io/projected/28f104e3-2da9-4ad6-bb98-361fd93879cd-kube-api-access-cnmsg\") pod \"ovnkube-node-7h6v4\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.730929 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.730903    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-hrf6t"]
Sep 10 13:20:27.734807 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.734791    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.747749 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.747730    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 13:20:27.747749 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.747740    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 13:20:27.747890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.747792    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lmg57\""
Sep 10 13:20:27.756925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.756903    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-sys\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.757003 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.756928    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-run\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.757003 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.756954    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-sysconfig\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.757003 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.756974    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-tmp\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.757097 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757032    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-kubernetes\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.757097 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757089    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4a5a18c4-1ee9-4a34-b304-49b3f6f26674-serviceca\") pod \"node-ca-l9cw9\" (UID: \"4a5a18c4-1ee9-4a34-b304-49b3f6f26674\") " pod="openshift-image-registry/node-ca-l9cw9"
Sep 10 13:20:27.757156 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757107    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-var-lib-kubelet\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.757188 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757160    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-lib-modules\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.757233 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757219    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr78x\" (UniqueName: \"kubernetes.io/projected/4a5a18c4-1ee9-4a34-b304-49b3f6f26674-kube-api-access-sr78x\") pod \"node-ca-l9cw9\" (UID: \"4a5a18c4-1ee9-4a34-b304-49b3f6f26674\") " pod="openshift-image-registry/node-ca-l9cw9"
Sep 10 13:20:27.757267 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757249    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-modprobe-d\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.757324 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757310    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a5a18c4-1ee9-4a34-b304-49b3f6f26674-host\") pod \"node-ca-l9cw9\" (UID: \"4a5a18c4-1ee9-4a34-b304-49b3f6f26674\") " pod="openshift-image-registry/node-ca-l9cw9"
Sep 10 13:20:27.757360 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757337    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-host\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.757399 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757364    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-sysctl-d\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.757399 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757383    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a5a18c4-1ee9-4a34-b304-49b3f6f26674-host\") pod \"node-ca-l9cw9\" (UID: \"4a5a18c4-1ee9-4a34-b304-49b3f6f26674\") " pod="openshift-image-registry/node-ca-l9cw9"
Sep 10 13:20:27.757399 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757388    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-systemd\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.757504 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757410    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-tuned\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.757504 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757442    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-sysctl-conf\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.757504 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757458    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk79m\" (UniqueName: \"kubernetes.io/projected/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-kube-api-access-wk79m\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.757504 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.757469    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4a5a18c4-1ee9-4a34-b304-49b3f6f26674-serviceca\") pod \"node-ca-l9cw9\" (UID: \"4a5a18c4-1ee9-4a34-b304-49b3f6f26674\") " pod="openshift-image-registry/node-ca-l9cw9"
Sep 10 13:20:27.810016 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.809998    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr78x\" (UniqueName: \"kubernetes.io/projected/4a5a18c4-1ee9-4a34-b304-49b3f6f26674-kube-api-access-sr78x\") pod \"node-ca-l9cw9\" (UID: \"4a5a18c4-1ee9-4a34-b304-49b3f6f26674\") " pod="openshift-image-registry/node-ca-l9cw9"
Sep 10 13:20:27.820177 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.820160    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:27.828782 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:20:27.828763    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f104e3_2da9_4ad6_bb98_361fd93879cd.slice/crio-2a5018238dfbc30e5296271323b6ab2540fae4f793143c7a24b9540da59ffdf7 WatchSource:0}: Error finding container 2a5018238dfbc30e5296271323b6ab2540fae4f793143c7a24b9540da59ffdf7: Status 404 returned error can't find the container with id 2a5018238dfbc30e5296271323b6ab2540fae4f793143c7a24b9540da59ffdf7
Sep 10 13:20:27.840196 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.840179    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6"
Sep 10 13:20:27.847659 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:20:27.847636    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4bbe93a_c0ac_43f2_bc2c_2509c66ff0bd.slice/crio-0aad2306215887e7095d911e56fe673805746c9ba1defd5f3d107cbbee4d956d WatchSource:0}: Error finding container 0aad2306215887e7095d911e56fe673805746c9ba1defd5f3d107cbbee4d956d: Status 404 returned error can't find the container with id 0aad2306215887e7095d911e56fe673805746c9ba1defd5f3d107cbbee4d956d
Sep 10 13:20:27.858559 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.858537    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-run\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.858643 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.858566    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:27.858680 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.858651    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:20:27.858712 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.858679    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-run\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.858712 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.858687    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-sysconfig\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.858775 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.858716    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-tmp\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.858775 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.858708    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-sysconfig\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.858775 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.858744    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs podName:8bc0fa69-2ca1-4cd6-b004-88863086d37d nodeName:}" failed. No retries permitted until 2026-09-10 13:20:28.858725582 +0000 UTC m=+139.677763969 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs") pod "network-metrics-daemon-xxmwh" (UID: "8bc0fa69-2ca1-4cd6-b004-88863086d37d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:20:27.858868 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.858788    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-kubernetes\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.858868 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.858821    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-var-lib-kubelet\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.858868 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.858850    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-kubernetes\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.858952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.858896    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-lib-modules\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.858952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.858935    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-var-lib-kubelet\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859020 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.858956    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-modprobe-d\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859020 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.858984    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-host\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859020 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.859016    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-host\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859161 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.859026    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-lib-modules\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859161 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.859038    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-sysctl-d\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859161 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.859076    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-systemd\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859161 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.859046    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-modprobe-d\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859161 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.859102    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-tuned\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859161 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.859105    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-sysctl-d\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859161 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.859131    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-sysctl-conf\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859161 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.859158    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk79m\" (UniqueName: \"kubernetes.io/projected/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-kube-api-access-wk79m\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.859126    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-systemd\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.859197    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-sys\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.859234    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-sysctl-conf\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.859451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.859268    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-sys\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.860855 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.860839    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-tmp\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.860922 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.860893    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-etc-tuned\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.927772 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.927752    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l9cw9"
Sep 10 13:20:27.934767 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.934740    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerStarted","Data":"2a5018238dfbc30e5296271323b6ab2540fae4f793143c7a24b9540da59ffdf7"}
Sep 10 13:20:27.935829 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.935808    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k4k7c" event={"ID":"ad5b6dca-ed23-4c99-8b04-68a1afae4e1f","Type":"ContainerStarted","Data":"5ce617793caff99d63ea0e6757005947e64aa387e0aea6d8ddc2ae85d9136e00"}
Sep 10 13:20:27.936544 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:20:27.936523    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a5a18c4_1ee9_4a34_b304_49b3f6f26674.slice/crio-49ab0405520d545ba4b87c97be8d6ac8c93b93f61cde2af058c0972fac70f1b1 WatchSource:0}: Error finding container 49ab0405520d545ba4b87c97be8d6ac8c93b93f61cde2af058c0972fac70f1b1: Status 404 returned error can't find the container with id 49ab0405520d545ba4b87c97be8d6ac8c93b93f61cde2af058c0972fac70f1b1
Sep 10 13:20:27.937695 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.937638    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" event={"ID":"3508d0a07aa0fd49c57a7f8e3fd46334","Type":"ContainerStarted","Data":"4983b0b5501e26cd1aa83fa54daaafbd43e76490789f26088349a39ef4bf414e"}
Sep 10 13:20:27.938727 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.938705    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6" event={"ID":"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd","Type":"ContainerStarted","Data":"0aad2306215887e7095d911e56fe673805746c9ba1defd5f3d107cbbee4d956d"}
Sep 10 13:20:27.943099 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.943081    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk79m\" (UniqueName: \"kubernetes.io/projected/0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7-kube-api-access-wk79m\") pod \"tuned-hrf6t\" (UID: \"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7\") " pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:27.960389 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.960369    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt7jx\" (UniqueName: \"kubernetes.io/projected/8bc0fa69-2ca1-4cd6-b004-88863086d37d-kube-api-access-vt7jx\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:27.960699 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.960410    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrm9s\" (UniqueName: \"kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s\") pod \"network-check-target-82cgh\" (UID: \"4c893996-bc6b-46f0-ab3f-b98dcb115908\") " pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:27.960821 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.960753    2551 projected.go:291] Couldn't get configMap openshift-multus/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.960821 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.960780    2551 projected.go:196] Error preparing data for projected volume kube-api-access-vt7jx for pod openshift-multus/network-metrics-daemon-xxmwh: configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.960930 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.960839    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bc0fa69-2ca1-4cd6-b004-88863086d37d-kube-api-access-vt7jx podName:8bc0fa69-2ca1-4cd6-b004-88863086d37d nodeName:}" failed. No retries permitted until 2026-09-10 13:20:28.960821977 +0000 UTC m=+139.779860373 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vt7jx" (UniqueName: "kubernetes.io/projected/8bc0fa69-2ca1-4cd6-b004-88863086d37d-kube-api-access-vt7jx") pod "network-metrics-daemon-xxmwh" (UID: "8bc0fa69-2ca1-4cd6-b004-88863086d37d") : configmap "kube-root-ca.crt" not found
Sep 10 13:20:27.960930 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.960909    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:20:27.960930 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.960930    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:20:27.961080 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.960970    2551 projected.go:196] Error preparing data for projected volume kube-api-access-xrm9s for pod openshift-network-diagnostics/network-check-target-82cgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:20:27.961080 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:27.961029    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s podName:4c893996-bc6b-46f0-ab3f-b98dcb115908 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:28.961007493 +0000 UTC m=+139.780045886 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrm9s" (UniqueName: "kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s") pod "network-check-target-82cgh" (UID: "4c893996-bc6b-46f0-ab3f-b98dcb115908") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:20:27.961826 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:27.961780    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-117.ec2.internal" podStartSLOduration=15.961758801 podStartE2EDuration="15.961758801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:27.961160644 +0000 UTC m=+138.780199044" watchObservedRunningTime="2026-09-10 13:20:27.961758801 +0000 UTC m=+138.780797201"
Sep 10 13:20:28.042465 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.042411    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hrf6t"
Sep 10 13:20:28.052133 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:20:28.052113    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fdefe0e_bbe6_4570_bfbb_5d39a93c55a7.slice/crio-662bd89a17e2927b284380ecaa1a58188e37506f79e12900d40bd6626243cdbe WatchSource:0}: Error finding container 662bd89a17e2927b284380ecaa1a58188e37506f79e12900d40bd6626243cdbe: Status 404 returned error can't find the container with id 662bd89a17e2927b284380ecaa1a58188e37506f79e12900d40bd6626243cdbe
Sep 10 13:20:28.061067 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.061048    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:28.061194 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:28.061176    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:20:28.061245 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:28.061236    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret podName:68de88a2-c141-4c07-9bfd-9ad2aa25b835 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:29.061224611 +0000 UTC m=+139.880262989 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret") pod "global-pull-secret-syncer-fnfwb" (UID: "68de88a2-c141-4c07-9bfd-9ad2aa25b835") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:20:28.120450 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.120407    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 13:15:26 +0000 UTC" deadline="2028-03-19 02:36:16.684299766 +0000 UTC"
Sep 10 13:20:28.120450 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.120442    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13333h15m48.563859666s"
Sep 10 13:20:28.362772 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.362681    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfg7m\" (UniqueName: \"kubernetes.io/projected/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-kube-api-access-jfg7m\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:28.365300 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.365272    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfg7m\" (UniqueName: \"kubernetes.io/projected/d37fb884-c078-4fd0-baf5-cd0ee01f2a7c-kube-api-access-jfg7m\") pod \"multus-additional-cni-plugins-ln5qf\" (UID: \"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c\") " pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:28.443313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.443273    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ln5qf"
Sep 10 13:20:28.452778 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:20:28.452728    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd37fb884_c078_4fd0_baf5_cd0ee01f2a7c.slice/crio-20b55c6b7a4b0507aa79e8f9310c2f8e0b514832ae1e59ece769b932202c6943 WatchSource:0}: Error finding container 20b55c6b7a4b0507aa79e8f9310c2f8e0b514832ae1e59ece769b932202c6943: Status 404 returned error can't find the container with id 20b55c6b7a4b0507aa79e8f9310c2f8e0b514832ae1e59ece769b932202c6943
Sep 10 13:20:28.664779 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.664691    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r47j\" (UniqueName: \"kubernetes.io/projected/5ec389cc-200a-4541-a957-8f419ee31f55-kube-api-access-5r47j\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:28.670965 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.670932    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r47j\" (UniqueName: \"kubernetes.io/projected/5ec389cc-200a-4541-a957-8f419ee31f55-kube-api-access-5r47j\") pod \"multus-mtb95\" (UID: \"5ec389cc-200a-4541-a957-8f419ee31f55\") " pod="openshift-multus/multus-mtb95"
Sep 10 13:20:28.685395 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.685370    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mtb95"
Sep 10 13:20:28.704069 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.704049    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:28.704185 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:28.704160    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:28.704252 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.704232    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:28.704330 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:28.704315    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:28.704387 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.704367    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:28.704486 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:28.704428    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:28.866178 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.866109    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:28.866360 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:28.866298    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:20:28.866360 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:28.866360    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs podName:8bc0fa69-2ca1-4cd6-b004-88863086d37d nodeName:}" failed. No retries permitted until 2026-09-10 13:20:30.866342108 +0000 UTC m=+141.685380491 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs") pod "network-metrics-daemon-xxmwh" (UID: "8bc0fa69-2ca1-4cd6-b004-88863086d37d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:20:28.945248 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.945161    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mtb95" event={"ID":"5ec389cc-200a-4541-a957-8f419ee31f55","Type":"ContainerStarted","Data":"2dd13ed733933fc674b438a11eb93117b6eb761676d8766bf237e61d7414da2d"}
Sep 10 13:20:28.949026 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.948995    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hrf6t" event={"ID":"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7","Type":"ContainerStarted","Data":"662bd89a17e2927b284380ecaa1a58188e37506f79e12900d40bd6626243cdbe"}
Sep 10 13:20:28.951646 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.951618    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ln5qf" event={"ID":"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c","Type":"ContainerStarted","Data":"20b55c6b7a4b0507aa79e8f9310c2f8e0b514832ae1e59ece769b932202c6943"}
Sep 10 13:20:28.954816 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.954786    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l9cw9" event={"ID":"4a5a18c4-1ee9-4a34-b304-49b3f6f26674","Type":"ContainerStarted","Data":"49ab0405520d545ba4b87c97be8d6ac8c93b93f61cde2af058c0972fac70f1b1"}
Sep 10 13:20:28.967362 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.967335    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt7jx\" (UniqueName: \"kubernetes.io/projected/8bc0fa69-2ca1-4cd6-b004-88863086d37d-kube-api-access-vt7jx\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:28.967469 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.967393    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrm9s\" (UniqueName: \"kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s\") pod \"network-check-target-82cgh\" (UID: \"4c893996-bc6b-46f0-ab3f-b98dcb115908\") " pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:28.967595 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:28.967563    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:20:28.967664 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:28.967603    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:20:28.967664 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:28.967617    2551 projected.go:196] Error preparing data for projected volume kube-api-access-xrm9s for pod openshift-network-diagnostics/network-check-target-82cgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:20:28.967781 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:28.967673    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s podName:4c893996-bc6b-46f0-ab3f-b98dcb115908 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:30.967654369 +0000 UTC m=+141.786692754 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrm9s" (UniqueName: "kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s") pod "network-check-target-82cgh" (UID: "4c893996-bc6b-46f0-ab3f-b98dcb115908") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:20:28.975424 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:28.975398    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt7jx\" (UniqueName: \"kubernetes.io/projected/8bc0fa69-2ca1-4cd6-b004-88863086d37d-kube-api-access-vt7jx\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:29.067898 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:29.067859    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:29.068077 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:29.068025    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:20:29.068151 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:29.068089    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret podName:68de88a2-c141-4c07-9bfd-9ad2aa25b835 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:31.068071862 +0000 UTC m=+141.887110253 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret") pod "global-pull-secret-syncer-fnfwb" (UID: "68de88a2-c141-4c07-9bfd-9ad2aa25b835") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:20:29.663895 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:29.663836    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:20:30.704556 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:30.704522    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:30.705131 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:30.704675    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:30.705131 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:30.705035    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:30.705248 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:30.705136    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:30.705248 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:30.705136    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:30.705248 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:30.705235    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:30.881890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:30.881363    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:30.881890 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:30.881530    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:20:30.881890 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:30.881614    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs podName:8bc0fa69-2ca1-4cd6-b004-88863086d37d nodeName:}" failed. No retries permitted until 2026-09-10 13:20:34.881595584 +0000 UTC m=+145.700633966 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs") pod "network-metrics-daemon-xxmwh" (UID: "8bc0fa69-2ca1-4cd6-b004-88863086d37d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:20:30.982113 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:30.982078    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrm9s\" (UniqueName: \"kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s\") pod \"network-check-target-82cgh\" (UID: \"4c893996-bc6b-46f0-ab3f-b98dcb115908\") " pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:30.982288 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:30.982252    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:20:30.982348 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:30.982298    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:20:30.982348 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:30.982311    2551 projected.go:196] Error preparing data for projected volume kube-api-access-xrm9s for pod openshift-network-diagnostics/network-check-target-82cgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:20:30.982437 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:30.982365    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s podName:4c893996-bc6b-46f0-ab3f-b98dcb115908 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:34.982347744 +0000 UTC m=+145.801386141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrm9s" (UniqueName: "kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s") pod "network-check-target-82cgh" (UID: "4c893996-bc6b-46f0-ab3f-b98dcb115908") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:20:31.083501 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:31.083460    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:31.083691 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:31.083639    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:20:31.083751 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:31.083700    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret podName:68de88a2-c141-4c07-9bfd-9ad2aa25b835 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:35.083683311 +0000 UTC m=+145.902721703 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret") pod "global-pull-secret-syncer-fnfwb" (UID: "68de88a2-c141-4c07-9bfd-9ad2aa25b835") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:20:32.704386 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:32.703874    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:32.704386 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:32.704015    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:32.704386 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:32.704073    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:32.704386 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:32.704150    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:32.704386 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:32.704206    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:32.704386 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:32.704349    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:34.665484 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:34.665395    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:20:34.704221 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:34.704157    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:34.704221 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:34.704177    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:34.704221 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:34.704157    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:34.704478 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:34.704283    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:34.704837 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:34.704687    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:34.704837 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:34.704786    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:34.912366 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:34.912331    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:34.912724 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:34.912568    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:20:34.912872 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:34.912732    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs podName:8bc0fa69-2ca1-4cd6-b004-88863086d37d nodeName:}" failed. No retries permitted until 2026-09-10 13:20:42.912691594 +0000 UTC m=+153.731729994 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs") pod "network-metrics-daemon-xxmwh" (UID: "8bc0fa69-2ca1-4cd6-b004-88863086d37d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:20:35.013018 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:35.012988    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrm9s\" (UniqueName: \"kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s\") pod \"network-check-target-82cgh\" (UID: \"4c893996-bc6b-46f0-ab3f-b98dcb115908\") " pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:35.013186 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:35.013142    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:20:35.013186 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:35.013157    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:20:35.013186 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:35.013166    2551 projected.go:196] Error preparing data for projected volume kube-api-access-xrm9s for pod openshift-network-diagnostics/network-check-target-82cgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:20:35.013367 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:35.013205    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s podName:4c893996-bc6b-46f0-ab3f-b98dcb115908 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:43.01319416 +0000 UTC m=+153.832232538 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrm9s" (UniqueName: "kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s") pod "network-check-target-82cgh" (UID: "4c893996-bc6b-46f0-ab3f-b98dcb115908") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:20:35.113500 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:35.113462    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:35.113954 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:35.113679    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:20:35.113954 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:35.113744    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret podName:68de88a2-c141-4c07-9bfd-9ad2aa25b835 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:43.113727595 +0000 UTC m=+153.932765974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret") pod "global-pull-secret-syncer-fnfwb" (UID: "68de88a2-c141-4c07-9bfd-9ad2aa25b835") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:20:36.703688 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:36.703653    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:36.703688 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:36.703674    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:36.704240 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:36.703652    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:36.704240 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:36.703794    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:36.704240 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:36.703862    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:36.704240 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:36.703922    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:38.703767 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:38.703732    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:38.704187 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:38.703732    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:38.704187 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:38.703861    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:38.704187 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:38.703732    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:38.704187 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:38.703931    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:38.704187 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:38.704002    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:38.727427 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:38.727395    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-7h6v4"]
Sep 10 13:20:39.666435 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:39.666398    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:20:40.704041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:40.704006    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:40.704453 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:40.704123    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:40.704453 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:40.704005    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:40.704453 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:40.704265    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:40.704453 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:40.704005    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:40.704453 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:40.704388    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:42.704362 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:42.704326    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:42.704874 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:42.704328    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:42.704874 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:42.704437    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:42.704874 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:42.704541    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:42.704874 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:42.704328    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:42.704874 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:42.704699    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:42.973824 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:42.973791    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:42.973990 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:42.973970    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:20:42.974061 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:42.974050    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs podName:8bc0fa69-2ca1-4cd6-b004-88863086d37d nodeName:}" failed. No retries permitted until 2026-09-10 13:20:58.974030915 +0000 UTC m=+169.793069297 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs") pod "network-metrics-daemon-xxmwh" (UID: "8bc0fa69-2ca1-4cd6-b004-88863086d37d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:20:43.074180 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:43.074146    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrm9s\" (UniqueName: \"kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s\") pod \"network-check-target-82cgh\" (UID: \"4c893996-bc6b-46f0-ab3f-b98dcb115908\") " pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:43.074341 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:43.074284    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:20:43.074341 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:43.074298    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:20:43.074341 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:43.074306    2551 projected.go:196] Error preparing data for projected volume kube-api-access-xrm9s for pod openshift-network-diagnostics/network-check-target-82cgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:20:43.074467 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:43.074365    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s podName:4c893996-bc6b-46f0-ab3f-b98dcb115908 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:59.074349436 +0000 UTC m=+169.893387817 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrm9s" (UniqueName: "kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s") pod "network-check-target-82cgh" (UID: "4c893996-bc6b-46f0-ab3f-b98dcb115908") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:20:43.174723 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:43.174692    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:43.174891 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:43.174868    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:20:43.174959 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:43.174953    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret podName:68de88a2-c141-4c07-9bfd-9ad2aa25b835 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:59.17493283 +0000 UTC m=+169.993971221 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret") pod "global-pull-secret-syncer-fnfwb" (UID: "68de88a2-c141-4c07-9bfd-9ad2aa25b835") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:20:44.667837 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:44.667676    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:20:44.703962 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:44.703942    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:44.704040 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:44.703990    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:44.704089 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:44.704038    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:44.704089 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:44.703941    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:44.704161 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:44.704106    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:44.704161 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:44.704145    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:45.000796 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.000758    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l9cw9" event={"ID":"4a5a18c4-1ee9-4a34-b304-49b3f6f26674","Type":"ContainerStarted","Data":"840c8bb825fd4583845871e5809c6416aeab4a852a689ff1bb603835e5f721c0"}
Sep 10 13:20:45.002680 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.002637    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6" event={"ID":"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd","Type":"ContainerStarted","Data":"35ea0279404bd421c66f20a963f5e6b01a3d974640adc7f27080269b521901fa"}
Sep 10 13:20:45.004190 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.004170    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mtb95" event={"ID":"5ec389cc-200a-4541-a957-8f419ee31f55","Type":"ContainerStarted","Data":"4da708d66d054e066240350f5a24e6f9c136519030a06c8fdccc51d55a471c05"}
Sep 10 13:20:45.005821 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.005752    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hrf6t" event={"ID":"0fdefe0e-bbe6-4570-bfbb-5d39a93c55a7","Type":"ContainerStarted","Data":"b43f86f21d7b13e97625a27ebb0a1533560f37940f6748c7e7d20bf8f1c9b7b4"}
Sep 10 13:20:45.009080 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.009056    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerStarted","Data":"99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4"}
Sep 10 13:20:45.009179 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.009115    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerStarted","Data":"0fed014842c901cabe7de92fbe77c71f7220e2b15617bfa6563fbd4f21c4f6e1"}
Sep 10 13:20:45.009179 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.009162    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerStarted","Data":"c065e04bf85a07432cd35109e029e45de1c70dc318ffd4bb074061368a7af67f"}
Sep 10 13:20:45.009284 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.009209    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerStarted","Data":"765d2b4e3fe4367f64549fd1aef0b2699661d6311e16a630d995edc907d2b7ec"}
Sep 10 13:20:45.009284 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.009257    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerStarted","Data":"fafafa2b0be7011e5befc42d412c9c73a373855c4ac8340d1ac200cf774cf526"}
Sep 10 13:20:45.009385 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.009306    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerStarted","Data":"dbfd55d99a97f95b80cdf3bcd468a54060e39fdf5bc7ee97053efd00135a76c0"}
Sep 10 13:20:45.010568 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.010547    2551 generic.go:356] "Generic (PLEG): container finished" podID="d37fb884-c078-4fd0-baf5-cd0ee01f2a7c" containerID="0e46b95e1f5696d5c65f46b06aca1bc764c5c8980e6702e9b57ec11cf1048da0" exitCode=0
Sep 10 13:20:45.010669 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.010603    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ln5qf" event={"ID":"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c","Type":"ContainerDied","Data":"0e46b95e1f5696d5c65f46b06aca1bc764c5c8980e6702e9b57ec11cf1048da0"}
Sep 10 13:20:45.016462 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.016412    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-l9cw9" podStartSLOduration=1.797217332 podStartE2EDuration="18.016394882s" totalImagesPullingTime="16.21917755s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:45.015927062 +0000 UTC m=+155.834965464" watchObservedRunningTime="2026-09-10 13:20:45.016394882 +0000 UTC m=+155.835433284"
Sep 10 13:20:45.059302 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.059259    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mtb95" podStartSLOduration=3.54363619 podStartE2EDuration="19.059246431s" totalImagesPullingTime="15.515610241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:45.058971369 +0000 UTC m=+155.878009769" watchObservedRunningTime="2026-09-10 13:20:45.059246431 +0000 UTC m=+155.878284878"
Sep 10 13:20:45.074133 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.074092    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hrf6t" podStartSLOduration=1.945816612 podStartE2EDuration="18.074079063s" totalImagesPullingTime="16.128262451s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:45.07391534 +0000 UTC m=+155.892953741" watchObservedRunningTime="2026-09-10 13:20:45.074079063 +0000 UTC m=+155.893117462"
Sep 10 13:20:45.423804 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.423653    2551 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 13:20:45.733898 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.733819    2551 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T13:20:45.423783505Z","UUID":"f23a5c9a-1f45-4aab-a6cd-900681a866a2","Handler":null,"Name":"","Endpoint":""}
Sep 10 13:20:45.735398 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.735367    2551 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 13:20:45.735398 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:45.735400    2551 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 13:20:46.015375 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:46.015291    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6" event={"ID":"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd","Type":"ContainerStarted","Data":"132637967ebaf3a61becd5452e7b139ce02977288477aead0078a2bd44e09ace"}
Sep 10 13:20:46.017254 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:46.017218    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k4k7c" event={"ID":"ad5b6dca-ed23-4c99-8b04-68a1afae4e1f","Type":"ContainerStarted","Data":"ff28934c7e74735b87f6fa80f5baf6f7a5ce4e28d1a0fee6e9eb10527043bf58"}
Sep 10 13:20:46.704125 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:46.703989    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:46.704269 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:46.704039    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:46.704269 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:46.704169    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:46.704269 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:46.704256    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:46.704400 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:46.704041    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:46.704400 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:46.704344    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:47.020819 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:47.020782    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6" event={"ID":"e4bbe93a-c0ac-43f2-bc2c-2509c66ff0bd","Type":"ContainerStarted","Data":"0c0fe37672339b25e41c5d622e096cac9a7c63b6df49f1c5952adcbbe7d6a513"}
Sep 10 13:20:47.023587 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:47.023548    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerStarted","Data":"e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7"}
Sep 10 13:20:47.051615 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:47.051550    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-k4k7c" podStartSLOduration=7.746097152 podStartE2EDuration="20.051536277s" totalImagesPullingTime="12.305439125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:46.033667267 +0000 UTC m=+156.852705666" watchObservedRunningTime="2026-09-10 13:20:47.051536277 +0000 UTC m=+157.870574730"
Sep 10 13:20:48.704134 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:48.704101    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:48.704134 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:48.704122    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:48.704790 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:48.704240    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:48.704790 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:48.704276    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:48.704790 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:48.704334    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:48.704790 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:48.704429    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:49.669189 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:49.669156    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:20:50.035060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.034821    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerStarted","Data":"a6541cb197e0c73123a504cf16c1612f1a52d9c62efd2dbbede843b1fd506033"}
Sep 10 13:20:50.035060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.035045    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="ovn-controller" containerID="cri-o://dbfd55d99a97f95b80cdf3bcd468a54060e39fdf5bc7ee97053efd00135a76c0" gracePeriod=30
Sep 10 13:20:50.035060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.035058    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="sbdb" containerID="cri-o://e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7" gracePeriod=30
Sep 10 13:20:50.035774 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.035085    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="ovn-acl-logging" containerID="cri-o://fafafa2b0be7011e5befc42d412c9c73a373855c4ac8340d1ac200cf774cf526" gracePeriod=30
Sep 10 13:20:50.035774 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.035119    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="kube-rbac-proxy-ovn-metrics" containerID="cri-o://c065e04bf85a07432cd35109e029e45de1c70dc318ffd4bb074061368a7af67f" gracePeriod=30
Sep 10 13:20:50.035774 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.035140    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="nbdb" containerID="cri-o://99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4" gracePeriod=30
Sep 10 13:20:50.035774 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.035079    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="kube-rbac-proxy-node" containerID="cri-o://765d2b4e3fe4367f64549fd1aef0b2699661d6311e16a630d995edc907d2b7ec" gracePeriod=30
Sep 10 13:20:50.035774 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.035209    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:50.035774 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.035280    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:20:50.035774 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.035211    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="northd" containerID="cri-o://0fed014842c901cabe7de92fbe77c71f7220e2b15617bfa6563fbd4f21c4f6e1" gracePeriod=30
Sep 10 13:20:50.037233 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:50.037196    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Sep 10 13:20:50.042054 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:50.041964    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Sep 10 13:20:50.049679 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:50.048202    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Sep 10 13:20:50.049679 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:50.048263    2551 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="sbdb" probeResult="unknown"
Sep 10 13:20:50.051984 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.051954    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="ovnkube-controller" containerID="cri-o://a6541cb197e0c73123a504cf16c1612f1a52d9c62efd2dbbede843b1fd506033" gracePeriod=30
Sep 10 13:20:50.062056 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.061757    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9m8l6" podStartSLOduration=4.723936946 podStartE2EDuration="23.06173836s" totalImagesPullingTime="18.337801414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:47.109450921 +0000 UTC m=+157.928489334" watchObservedRunningTime="2026-09-10 13:20:50.06173836 +0000 UTC m=+160.880776761"
Sep 10 13:20:50.703772 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.703686    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:50.703772 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.703719    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:50.703981 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:50.703686    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:50.703981 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:50.703817    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:50.703981 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:50.703928    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:50.704120 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:50.703991    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:51.039748 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.039725    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/kube-rbac-proxy-ovn-metrics/0.log"
Sep 10 13:20:51.040152 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.040137    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/kube-rbac-proxy-node/0.log"
Sep 10 13:20:51.040532 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.040518    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/ovn-acl-logging/0.log"
Sep 10 13:20:51.040941 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.040927    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/ovn-controller/0.log"
Sep 10 13:20:51.040996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.040960    2551 generic.go:356] "Generic (PLEG): container finished" podID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerID="e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7" exitCode=0
Sep 10 13:20:51.040996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.040973    2551 generic.go:356] "Generic (PLEG): container finished" podID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerID="99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4" exitCode=0
Sep 10 13:20:51.040996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.040979    2551 generic.go:356] "Generic (PLEG): container finished" podID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerID="0fed014842c901cabe7de92fbe77c71f7220e2b15617bfa6563fbd4f21c4f6e1" exitCode=0
Sep 10 13:20:51.040996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.040984    2551 generic.go:356] "Generic (PLEG): container finished" podID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerID="c065e04bf85a07432cd35109e029e45de1c70dc318ffd4bb074061368a7af67f" exitCode=143
Sep 10 13:20:51.040996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.040989    2551 generic.go:356] "Generic (PLEG): container finished" podID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerID="765d2b4e3fe4367f64549fd1aef0b2699661d6311e16a630d995edc907d2b7ec" exitCode=143
Sep 10 13:20:51.040996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.040994    2551 generic.go:356] "Generic (PLEG): container finished" podID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerID="fafafa2b0be7011e5befc42d412c9c73a373855c4ac8340d1ac200cf774cf526" exitCode=143
Sep 10 13:20:51.041197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.040999    2551 generic.go:356] "Generic (PLEG): container finished" podID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerID="dbfd55d99a97f95b80cdf3bcd468a54060e39fdf5bc7ee97053efd00135a76c0" exitCode=143
Sep 10 13:20:51.041197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.041031    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerDied","Data":"e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7"}
Sep 10 13:20:51.041197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.041100    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerDied","Data":"99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4"}
Sep 10 13:20:51.041197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.041133    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerDied","Data":"0fed014842c901cabe7de92fbe77c71f7220e2b15617bfa6563fbd4f21c4f6e1"}
Sep 10 13:20:51.041197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.041168    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerDied","Data":"c065e04bf85a07432cd35109e029e45de1c70dc318ffd4bb074061368a7af67f"}
Sep 10 13:20:51.041197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.041196    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerDied","Data":"765d2b4e3fe4367f64549fd1aef0b2699661d6311e16a630d995edc907d2b7ec"}
Sep 10 13:20:51.041408 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.041231    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerDied","Data":"fafafa2b0be7011e5befc42d412c9c73a373855c4ac8340d1ac200cf774cf526"}
Sep 10 13:20:51.041408 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.041275    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerDied","Data":"dbfd55d99a97f95b80cdf3bcd468a54060e39fdf5bc7ee97053efd00135a76c0"}
Sep 10 13:20:51.042778 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.042761    2551 generic.go:356] "Generic (PLEG): container finished" podID="d37fb884-c078-4fd0-baf5-cd0ee01f2a7c" containerID="a2809749bc4b90d4d3f83fe2b32d8f72b06e6a75388f0d8424b669a36d32eaab" exitCode=0
Sep 10 13:20:51.042873 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.042791    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ln5qf" event={"ID":"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c","Type":"ContainerDied","Data":"a2809749bc4b90d4d3f83fe2b32d8f72b06e6a75388f0d8424b669a36d32eaab"}
Sep 10 13:20:51.062557 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:51.062513    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podStartSLOduration=7.69191747 podStartE2EDuration="24.062501245s" totalImagesPullingTime="16.370583775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:50.062027469 +0000 UTC m=+160.881065869" watchObservedRunningTime="2026-09-10 13:20:51.062501245 +0000 UTC m=+161.881539644"
Sep 10 13:20:52.703930 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:52.703882    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:52.703930 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:52.703919    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:52.704327 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:52.704008    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:52.704327 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:52.704063    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:52.704327 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:52.704155    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:52.704327 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:52.704213    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:54.051069 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:54.051037    2551 generic.go:356] "Generic (PLEG): container finished" podID="d37fb884-c078-4fd0-baf5-cd0ee01f2a7c" containerID="d26eb96e093ce950a135dfba08457f749f02b883e1bd148c7a0340266223a2f1" exitCode=0
Sep 10 13:20:54.051424 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:54.051078    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ln5qf" event={"ID":"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c","Type":"ContainerDied","Data":"d26eb96e093ce950a135dfba08457f749f02b883e1bd148c7a0340266223a2f1"}
Sep 10 13:20:54.669946 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:54.669912    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:20:54.704386 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:54.704336    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:54.704386 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:54.704357    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:54.704636 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:54.704450    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:54.704636 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:54.704465    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:54.704636 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:54.704531    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:54.704636 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:54.704598    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:56.058130 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:56.058099    2551 generic.go:356] "Generic (PLEG): container finished" podID="d37fb884-c078-4fd0-baf5-cd0ee01f2a7c" containerID="4b460154f956a35444264b31729e4b818358e13e696aac860a4b2de73f5a7316" exitCode=0
Sep 10 13:20:56.058612 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:56.058137    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ln5qf" event={"ID":"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c","Type":"ContainerDied","Data":"4b460154f956a35444264b31729e4b818358e13e696aac860a4b2de73f5a7316"}
Sep 10 13:20:56.703973 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:56.703945    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:56.704137 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:56.703945    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:56.704137 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:56.704046    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:56.704224 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:56.704150    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:56.704224 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:56.703945    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:56.704312 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:56.704244    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:58.703798 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.703769    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:58.703798 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.703793    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:58.704227 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.703769    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:58.704227 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:58.703940    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:20:58.704227 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:58.704043    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:20:58.704227 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:58.704145    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:20:58.771182 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.771154    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-cz6jf"]
Sep 10 13:20:58.794927 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.794897    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cz6jf"
Sep 10 13:20:58.797514 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.797492    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6m2vk\""
Sep 10 13:20:58.797656 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.797537    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 13:20:58.798366 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.798347    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 13:20:58.876844 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.876662    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ae6bfe75-19cb-49df-978c-1f369f73429f-hosts-file\") pod \"node-resolver-cz6jf\" (UID: \"ae6bfe75-19cb-49df-978c-1f369f73429f\") " pod="openshift-dns/node-resolver-cz6jf"
Sep 10 13:20:58.876997 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.876883    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae6bfe75-19cb-49df-978c-1f369f73429f-tmp-dir\") pod \"node-resolver-cz6jf\" (UID: \"ae6bfe75-19cb-49df-978c-1f369f73429f\") " pod="openshift-dns/node-resolver-cz6jf"
Sep 10 13:20:58.876997 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.876912    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2nd2\" (UniqueName: \"kubernetes.io/projected/ae6bfe75-19cb-49df-978c-1f369f73429f-kube-api-access-w2nd2\") pod \"node-resolver-cz6jf\" (UID: \"ae6bfe75-19cb-49df-978c-1f369f73429f\") " pod="openshift-dns/node-resolver-cz6jf"
Sep 10 13:20:58.978209 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.978180    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae6bfe75-19cb-49df-978c-1f369f73429f-tmp-dir\") pod \"node-resolver-cz6jf\" (UID: \"ae6bfe75-19cb-49df-978c-1f369f73429f\") " pod="openshift-dns/node-resolver-cz6jf"
Sep 10 13:20:58.978357 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.978216    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:20:58.978357 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.978235    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2nd2\" (UniqueName: \"kubernetes.io/projected/ae6bfe75-19cb-49df-978c-1f369f73429f-kube-api-access-w2nd2\") pod \"node-resolver-cz6jf\" (UID: \"ae6bfe75-19cb-49df-978c-1f369f73429f\") " pod="openshift-dns/node-resolver-cz6jf"
Sep 10 13:20:58.978357 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.978287    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ae6bfe75-19cb-49df-978c-1f369f73429f-hosts-file\") pod \"node-resolver-cz6jf\" (UID: \"ae6bfe75-19cb-49df-978c-1f369f73429f\") " pod="openshift-dns/node-resolver-cz6jf"
Sep 10 13:20:58.978470 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:58.978394    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:20:58.978470 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:58.978461    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs podName:8bc0fa69-2ca1-4cd6-b004-88863086d37d nodeName:}" failed. No retries permitted until 2026-09-10 13:21:30.978446478 +0000 UTC m=+201.797484875 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs") pod "network-metrics-daemon-xxmwh" (UID: "8bc0fa69-2ca1-4cd6-b004-88863086d37d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:20:58.978549 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.978478    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ae6bfe75-19cb-49df-978c-1f369f73429f-hosts-file\") pod \"node-resolver-cz6jf\" (UID: \"ae6bfe75-19cb-49df-978c-1f369f73429f\") " pod="openshift-dns/node-resolver-cz6jf"
Sep 10 13:20:58.978549 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.978530    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae6bfe75-19cb-49df-978c-1f369f73429f-tmp-dir\") pod \"node-resolver-cz6jf\" (UID: \"ae6bfe75-19cb-49df-978c-1f369f73429f\") " pod="openshift-dns/node-resolver-cz6jf"
Sep 10 13:20:58.988994 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:58.988964    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2nd2\" (UniqueName: \"kubernetes.io/projected/ae6bfe75-19cb-49df-978c-1f369f73429f-kube-api-access-w2nd2\") pod \"node-resolver-cz6jf\" (UID: \"ae6bfe75-19cb-49df-978c-1f369f73429f\") " pod="openshift-dns/node-resolver-cz6jf"
Sep 10 13:20:59.079135 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:59.079101    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrm9s\" (UniqueName: \"kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s\") pod \"network-check-target-82cgh\" (UID: \"4c893996-bc6b-46f0-ab3f-b98dcb115908\") " pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:20:59.079323 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:59.079278    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:20:59.079323 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:59.079302    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:20:59.079323 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:59.079316    2551 projected.go:196] Error preparing data for projected volume kube-api-access-xrm9s for pod openshift-network-diagnostics/network-check-target-82cgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:20:59.079484 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:59.079380    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s podName:4c893996-bc6b-46f0-ab3f-b98dcb115908 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:31.079363041 +0000 UTC m=+201.898401422 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrm9s" (UniqueName: "kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s") pod "network-check-target-82cgh" (UID: "4c893996-bc6b-46f0-ab3f-b98dcb115908") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:20:59.104946 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:59.104915    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cz6jf"
Sep 10 13:20:59.179881 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:20:59.179844    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:20:59.180057 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:59.180033    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:20:59.180134 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:59.180109    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret podName:68de88a2-c141-4c07-9bfd-9ad2aa25b835 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:31.180091021 +0000 UTC m=+201.999129419 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret") pod "global-pull-secret-syncer-fnfwb" (UID: "68de88a2-c141-4c07-9bfd-9ad2aa25b835") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:20:59.670912 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:20:59.670878    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:21:00.035996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:00.035962    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:21:00.036517 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:00.036375    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4 is running failed: container process not found" containerID="99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Sep 10 13:21:00.036517 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:00.036365    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7 is running failed: container process not found" containerID="e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Sep 10 13:21:00.036795 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:00.036763    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4 is running failed: container process not found" containerID="99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Sep 10 13:21:00.036865 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:00.036770    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7 is running failed: container process not found" containerID="e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Sep 10 13:21:00.037095 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:00.037064    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7 is running failed: container process not found" containerID="e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Sep 10 13:21:00.037203 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:00.037109    2551 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7 is running failed: container process not found" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="sbdb" probeResult="unknown"
Sep 10 13:21:00.037203 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:00.037141    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4 is running failed: container process not found" containerID="99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Sep 10 13:21:00.037203 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:00.037182    2551 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4 is running failed: container process not found" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="nbdb" probeResult="unknown"
Sep 10 13:21:00.478311 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:21:00.478279    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae6bfe75_19cb_49df_978c_1f369f73429f.slice/crio-b851422b0662c17eb48c23fcea47d9bf4edbcd30dc958e439ab68f54d17bad1d WatchSource:0}: Error finding container b851422b0662c17eb48c23fcea47d9bf4edbcd30dc958e439ab68f54d17bad1d: Status 404 returned error can't find the container with id b851422b0662c17eb48c23fcea47d9bf4edbcd30dc958e439ab68f54d17bad1d
Sep 10 13:21:00.704279 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:00.704244    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:00.704407 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:00.704282    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:00.704407 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:00.704347    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:00.704407 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:00.704349    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:00.704531 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:00.704438    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:00.704569 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:00.704532    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:01.072369 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:01.072297    2551 generic.go:356] "Generic (PLEG): container finished" podID="d37fb884-c078-4fd0-baf5-cd0ee01f2a7c" containerID="f0d93b4cdbe8b2a6f23811284c766dd2a1132e983de1b4a6889bc97f3eeacc67" exitCode=0
Sep 10 13:21:01.072907 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:01.072381    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ln5qf" event={"ID":"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c","Type":"ContainerDied","Data":"f0d93b4cdbe8b2a6f23811284c766dd2a1132e983de1b4a6889bc97f3eeacc67"}
Sep 10 13:21:01.073841 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:01.073821    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cz6jf" event={"ID":"ae6bfe75-19cb-49df-978c-1f369f73429f","Type":"ContainerStarted","Data":"6300d7a7ba89633879b5e17bf8b5682d32c4c5be783d7b25b748a44faf8a5ecc"}
Sep 10 13:21:01.073945 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:01.073856    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cz6jf" event={"ID":"ae6bfe75-19cb-49df-978c-1f369f73429f","Type":"ContainerStarted","Data":"b851422b0662c17eb48c23fcea47d9bf4edbcd30dc958e439ab68f54d17bad1d"}
Sep 10 13:21:02.078387 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:02.078281    2551 generic.go:356] "Generic (PLEG): container finished" podID="d37fb884-c078-4fd0-baf5-cd0ee01f2a7c" containerID="8e05d475d69c935c7a712cad14bacfd359145b8855efef3e1163f8fffec0796a" exitCode=0
Sep 10 13:21:02.078387 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:02.078329    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ln5qf" event={"ID":"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c","Type":"ContainerDied","Data":"8e05d475d69c935c7a712cad14bacfd359145b8855efef3e1163f8fffec0796a"}
Sep 10 13:21:02.101845 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:02.101791    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cz6jf" podStartSLOduration=4.101777386 podStartE2EDuration="4.101777386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:01.128628784 +0000 UTC m=+171.947667185" watchObservedRunningTime="2026-09-10 13:21:02.101777386 +0000 UTC m=+172.920815845"
Sep 10 13:21:02.704222 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:02.704188    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:02.704379 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:02.704194    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:02.704379 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:02.704292    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:02.704379 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:02.704194    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:02.704488 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:02.704378    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:02.704488 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:02.704453    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:03.084284 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:03.084250    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ln5qf" event={"ID":"d37fb884-c078-4fd0-baf5-cd0ee01f2a7c","Type":"ContainerStarted","Data":"97a47e6e0016c6c39b3854c68c0277dc2a88074e06dd7e347262269a8556bea6"}
Sep 10 13:21:04.672321 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:04.672287    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:21:04.704539 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:04.704517    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:04.704635 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:04.704518    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:04.704635 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:04.704517    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:04.704698 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:04.704657    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:04.704759 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:04.704739    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:04.704811 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:04.704802    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:06.704319 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:06.704273    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:06.704725 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:06.704273    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:06.704725 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:06.704397    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:06.704725 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:06.704286    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:06.704725 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:06.704469    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:06.704725 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:06.704547    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:08.703652 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:08.703615    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:08.704101 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:08.703661    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:08.704101 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:08.703709    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:08.704101 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:08.703723    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:08.704101 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:08.703779    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:08.704101 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:08.703887    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:09.673680 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:09.673643    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:21:10.035655 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:10.035617    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7 is running failed: container process not found" containerID="e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Sep 10 13:21:10.036118 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:10.035615    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4 is running failed: container process not found" containerID="99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Sep 10 13:21:10.036118 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:10.035911    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4 is running failed: container process not found" containerID="99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Sep 10 13:21:10.036118 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:10.035940    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7 is running failed: container process not found" containerID="e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Sep 10 13:21:10.036236 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:10.036183    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7 is running failed: container process not found" containerID="e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Sep 10 13:21:10.036236 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:10.036218    2551 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7 is running failed: container process not found" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="sbdb" probeResult="unknown"
Sep 10 13:21:10.036299 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:10.036184    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4 is running failed: container process not found" containerID="99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Sep 10 13:21:10.036299 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:10.036275    2551 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4 is running failed: container process not found" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="nbdb" probeResult="unknown"
Sep 10 13:21:10.704410 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:10.704376    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:10.704561 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:10.704383    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:10.704561 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:10.704492    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:10.704561 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:10.704517    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:10.704692 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:10.704570    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:10.704692 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:10.704635    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:12.704096 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:12.704060    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:12.704490 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:12.704099    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:12.704490 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:12.704163    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:12.704490 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:12.704186    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:12.704490 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:12.704259    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:12.704490 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:12.704323    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:14.674682 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:14.674642    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:21:14.704050 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:14.704012    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:14.704167 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:14.704060    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:14.704167 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:14.704116    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:14.704167 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:14.704135    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:14.704277 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:14.704189    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:14.704277 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:14.704253    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:16.704614 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:16.704564    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:16.705135 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:16.704564    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:16.705135 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:16.704709    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:16.705135 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:16.704781    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:16.705135 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:16.704568    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:16.705135 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:16.704847    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:18.703980 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:18.703949    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:18.704329 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:18.703950    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:18.704329 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:18.704058    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:18.704329 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:18.703950    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:18.704329 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:18.704184    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:18.704329 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:18.704117    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:19.092005 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:19.091979    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cz6jf_ae6bfe75-19cb-49df-978c-1f369f73429f/dns-node-resolver/0.log"
Sep 10 13:21:19.675037 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:19.675006    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:21:20.003516 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.003470    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l9cw9_4a5a18c4-1ee9-4a34-b304-49b3f6f26674/node-ca/0.log"
Sep 10 13:21:20.036186 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.036152    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7 is running failed: container process not found" containerID="e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Sep 10 13:21:20.036186 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.036159    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4 is running failed: container process not found" containerID="99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Sep 10 13:21:20.036491 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.036469    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7 is running failed: container process not found" containerID="e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Sep 10 13:21:20.036602 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.036466    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4 is running failed: container process not found" containerID="99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Sep 10 13:21:20.036832 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.036797    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4 is running failed: container process not found" containerID="99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Sep 10 13:21:20.036928 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.036841    2551 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4 is running failed: container process not found" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="nbdb" probeResult="unknown"
Sep 10 13:21:20.036928 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.036810    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7 is running failed: container process not found" containerID="e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Sep 10 13:21:20.036928 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.036917    2551 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7 is running failed: container process not found" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="sbdb" probeResult="unknown"
Sep 10 13:21:20.037308 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.037287    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a6541cb197e0c73123a504cf16c1612f1a52d9c62efd2dbbede843b1fd506033" cmd=["test","-f","/etc/cni/net.d/10-ovn-kubernetes.conf"]
Sep 10 13:21:20.038547 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.038525    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a6541cb197e0c73123a504cf16c1612f1a52d9c62efd2dbbede843b1fd506033" cmd=["test","-f","/etc/cni/net.d/10-ovn-kubernetes.conf"]
Sep 10 13:21:20.039334 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.039308    2551 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a6541cb197e0c73123a504cf16c1612f1a52d9c62efd2dbbede843b1fd506033" cmd=["test","-f","/etc/cni/net.d/10-ovn-kubernetes.conf"]
Sep 10 13:21:20.039406 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.039347    2551 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="ovnkube-controller" probeResult="unknown"
Sep 10 13:21:20.123065 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.122937    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/ovnkube-controller/0.log"
Sep 10 13:21:20.124310 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.124295    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/kube-rbac-proxy-ovn-metrics/0.log"
Sep 10 13:21:20.124681 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.124666    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/kube-rbac-proxy-node/0.log"
Sep 10 13:21:20.125047 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.125033    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/ovn-acl-logging/0.log"
Sep 10 13:21:20.125427 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.125415    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/ovn-controller/0.log"
Sep 10 13:21:20.125484 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.125446    2551 generic.go:356] "Generic (PLEG): container finished" podID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerID="a6541cb197e0c73123a504cf16c1612f1a52d9c62efd2dbbede843b1fd506033" exitCode=137
Sep 10 13:21:20.125484 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.125474    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerDied","Data":"a6541cb197e0c73123a504cf16c1612f1a52d9c62efd2dbbede843b1fd506033"}
Sep 10 13:21:20.586660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.586643    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/ovnkube-controller/0.log"
Sep 10 13:21:20.587969 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.587951    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/kube-rbac-proxy-ovn-metrics/0.log"
Sep 10 13:21:20.588371 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.588357    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/kube-rbac-proxy-node/0.log"
Sep 10 13:21:20.588756 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.588740    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/ovn-acl-logging/0.log"
Sep 10 13:21:20.589130 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.589118    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/ovn-controller/0.log"
Sep 10 13:21:20.611944 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.611926    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:21:20.641990 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.641951    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ln5qf" podStartSLOduration=25.793794668 podStartE2EDuration="54.64193916s" totalImagesPullingTime="28.848144492s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:03.110924618 +0000 UTC m=+173.929963016" watchObservedRunningTime="2026-09-10 13:21:20.64193916 +0000 UTC m=+191.460977592"
Sep 10 13:21:20.672335 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672317    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-fc2l7"]
Sep 10 13:21:20.672463 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672451    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="kube-rbac-proxy-ovn-metrics"
Sep 10 13:21:20.672502 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672467    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="kube-rbac-proxy-ovn-metrics"
Sep 10 13:21:20.672502 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672478    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="ovn-controller"
Sep 10 13:21:20.672502 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672484    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="ovn-controller"
Sep 10 13:21:20.672502 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672491    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="sbdb"
Sep 10 13:21:20.672502 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672497    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="sbdb"
Sep 10 13:21:20.672502 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672502    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="nbdb"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672507    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="nbdb"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672517    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="kube-rbac-proxy-node"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672522    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="kube-rbac-proxy-node"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672548    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="ovnkube-controller"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672557    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="ovnkube-controller"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672565    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="ovn-acl-logging"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672582    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="ovn-acl-logging"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672587    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="northd"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672594    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="northd"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672621    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="sbdb"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672631    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="ovn-controller"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672639    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="nbdb"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672645    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="kube-rbac-proxy-node"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672650    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="kube-rbac-proxy-ovn-metrics"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672654    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="northd"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672659    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="ovnkube-controller"
Sep 10 13:21:20.672706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.672664    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" containerName="ovn-acl-logging"
Sep 10 13:21:20.690345 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.690329    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.704118 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.704101    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:20.704118 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.704112    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:20.704235 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.704114    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:20.704235 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.704173    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:20.704338 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.704299    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:20.704421 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:20.704401    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:20.709209 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709194    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-systemd-units\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.709271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709215    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-cni-bin\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.709271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709233    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-slash\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.709271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709246    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-run-ovn-kubernetes\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.709271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709259    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-cni-netd\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.709451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709278    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-node-log\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.709451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709286    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-cni-bin" (OuterVolumeSpecName: "host-cni-bin") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "host-cni-bin". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.709451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709305    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnmsg\" (UniqueName: \"kubernetes.io/projected/28f104e3-2da9-4ad6-bb98-361fd93879cd-kube-api-access-cnmsg\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.709451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709306    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-systemd-units" (OuterVolumeSpecName: "systemd-units") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "systemd-units". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.709451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709286    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-slash" (OuterVolumeSpecName: "host-slash") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "host-slash". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.709451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709310    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-cni-netd" (OuterVolumeSpecName: "host-cni-netd") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "host-cni-netd". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.709451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709325    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-run-ovn-kubernetes" (OuterVolumeSpecName: "host-run-ovn-kubernetes") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "host-run-ovn-kubernetes". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.709451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709336    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-node-log" (OuterVolumeSpecName: "node-log") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "node-log". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.709451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709341    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-kubelet\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.709451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709361    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-kubelet" (OuterVolumeSpecName: "host-kubelet") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "host-kubelet". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.709451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709402    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovnkube-script-lib\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.709451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709421    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-ovn\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.709451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709444    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-openvswitch\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709472    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-ovn" (OuterVolumeSpecName: "run-ovn") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "run-ovn". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709475    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-openvswitch" (OuterVolumeSpecName: "run-openvswitch") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "run-openvswitch". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709491    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-etc-openvswitch\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709510    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-run-netns\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709526    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovnkube-config\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709546    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-env-overrides\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709557    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-run-netns" (OuterVolumeSpecName: "host-run-netns") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "host-run-netns". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709569    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-systemd\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709594    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-etc-openvswitch" (OuterVolumeSpecName: "etc-openvswitch") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "etc-openvswitch". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709615    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709649    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovn-node-metrics-cert\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709674    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-log-socket\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709687    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-var-lib-cni-networks-ovn-kubernetes" (OuterVolumeSpecName: "host-var-lib-cni-networks-ovn-kubernetes") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "host-var-lib-cni-networks-ovn-kubernetes". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709698    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-var-lib-openvswitch\") pod \"28f104e3-2da9-4ad6-bb98-361fd93879cd\" (UID: \"28f104e3-2da9-4ad6-bb98-361fd93879cd\") "
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709747    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovnkube-script-lib" (OuterVolumeSpecName: "ovnkube-script-lib") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "ovnkube-script-lib". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:21:20.710064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709774    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-log-socket" (OuterVolumeSpecName: "log-socket") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "log-socket". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709782    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-var-lib-openvswitch" (OuterVolumeSpecName: "var-lib-openvswitch") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "var-lib-openvswitch". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709861    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovnkube-config" (OuterVolumeSpecName: "ovnkube-config") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "ovnkube-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709870    2551 reconciler_common.go:299] "Volume detached for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-log-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709886    2551 reconciler_common.go:299] "Volume detached for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-var-lib-openvswitch\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709895    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-env-overrides" (OuterVolumeSpecName: "env-overrides") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "env-overrides". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709897    2551 reconciler_common.go:299] "Volume detached for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-systemd-units\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709925    2551 reconciler_common.go:299] "Volume detached for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-cni-bin\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709935    2551 reconciler_common.go:299] "Volume detached for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-slash\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709943    2551 reconciler_common.go:299] "Volume detached for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-run-ovn-kubernetes\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709952    2551 reconciler_common.go:299] "Volume detached for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-cni-netd\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709962    2551 reconciler_common.go:299] "Volume detached for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-node-log\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709970    2551 reconciler_common.go:299] "Volume detached for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-kubelet\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709980    2551 reconciler_common.go:299] "Volume detached for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovnkube-script-lib\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.709991    2551 reconciler_common.go:299] "Volume detached for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-ovn\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.710000    2551 reconciler_common.go:299] "Volume detached for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-openvswitch\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.710008    2551 reconciler_common.go:299] "Volume detached for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-etc-openvswitch\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.710016    2551 reconciler_common.go:299] "Volume detached for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-run-netns\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.710890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.710026    2551 reconciler_common.go:299] "Volume detached for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-host-var-lib-cni-networks-ovn-kubernetes\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.712012 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.711987    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovn-node-metrics-cert" (OuterVolumeSpecName: "ovn-node-metrics-cert") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "ovn-node-metrics-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:21:20.712068 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.711987    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28f104e3-2da9-4ad6-bb98-361fd93879cd-kube-api-access-cnmsg" (OuterVolumeSpecName: "kube-api-access-cnmsg") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "kube-api-access-cnmsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:21:20.712901 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.712883    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-systemd" (OuterVolumeSpecName: "run-systemd") pod "28f104e3-2da9-4ad6-bb98-361fd93879cd" (UID: "28f104e3-2da9-4ad6-bb98-361fd93879cd"). InnerVolumeSpecName "run-systemd". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 10 13:21:20.810473 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810448    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-run-systemd\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.810589 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810477    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-log-socket\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.810589 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810495    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-kubelet\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.810589 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810510    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-run-openvswitch\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.810589 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810525    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-run-ovn\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.810589 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810539    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-cni-bin\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.810806 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810621    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-run-netns\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.810806 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810662    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0555c0a2-fbef-4218-8f46-7beab92fb196-ovnkube-script-lib\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.810806 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810681    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0555c0a2-fbef-4218-8f46-7beab92fb196-ovnkube-config\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.810806 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810706    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-run-ovn-kubernetes\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.810806 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810731    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q68w\" (UniqueName: \"kubernetes.io/projected/0555c0a2-fbef-4218-8f46-7beab92fb196-kube-api-access-8q68w\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.810806 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810758    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-systemd-units\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.810806 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810777    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-slash\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.811029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810810    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-etc-openvswitch\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.811029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810829    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0555c0a2-fbef-4218-8f46-7beab92fb196-ovn-node-metrics-cert\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.811029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810850    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.811029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810895    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-var-lib-openvswitch\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.811029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810911    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-node-log\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.811029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810927    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-cni-netd\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.811029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810941    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0555c0a2-fbef-4218-8f46-7beab92fb196-env-overrides\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.811029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810984    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnmsg\" (UniqueName: \"kubernetes.io/projected/28f104e3-2da9-4ad6-bb98-361fd93879cd-kube-api-access-cnmsg\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.811029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.810996    2551 reconciler_common.go:299] "Volume detached for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovnkube-config\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.811029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.811005    2551 reconciler_common.go:299] "Volume detached for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/28f104e3-2da9-4ad6-bb98-361fd93879cd-env-overrides\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.811029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.811018    2551 reconciler_common.go:299] "Volume detached for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/28f104e3-2da9-4ad6-bb98-361fd93879cd-run-systemd\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.811029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.811028    2551 reconciler_common.go:299] "Volume detached for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/28f104e3-2da9-4ad6-bb98-361fd93879cd-ovn-node-metrics-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:21:20.911754 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911680    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q68w\" (UniqueName: \"kubernetes.io/projected/0555c0a2-fbef-4218-8f46-7beab92fb196-kube-api-access-8q68w\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.911754 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911719    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-systemd-units\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.911754 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911738    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-slash\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.911913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911762    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-etc-openvswitch\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.911913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911786    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0555c0a2-fbef-4218-8f46-7beab92fb196-ovn-node-metrics-cert\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.911913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911790    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-systemd-units\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.911913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911801    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-slash\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.911913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911810    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.911913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911832    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-etc-openvswitch\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.911913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911851    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.911913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911861    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-var-lib-openvswitch\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.911913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911880    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-node-log\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.911913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911896    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-cni-netd\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.911913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911911    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0555c0a2-fbef-4218-8f46-7beab92fb196-env-overrides\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911955    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-node-log\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911965    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-cni-netd\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911981    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-var-lib-openvswitch\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.911986    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-run-systemd\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912016    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-run-systemd\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912037    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-log-socket\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912068    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-kubelet\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912097    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-run-openvswitch\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912125    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-kubelet\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912127    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-run-ovn\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912143    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-log-socket\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912160    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-cni-bin\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912198    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-cni-bin\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912200    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-run-ovn\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912200    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-run-netns\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912160    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-run-openvswitch\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912229    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-run-netns\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912257    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0555c0a2-fbef-4218-8f46-7beab92fb196-ovnkube-script-lib\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912917 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912308    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0555c0a2-fbef-4218-8f46-7beab92fb196-ovnkube-config\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912917 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912339    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-run-ovn-kubernetes\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912917 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912388    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0555c0a2-fbef-4218-8f46-7beab92fb196-env-overrides\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912917 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912397    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0555c0a2-fbef-4218-8f46-7beab92fb196-host-run-ovn-kubernetes\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912917 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912685    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0555c0a2-fbef-4218-8f46-7beab92fb196-ovnkube-script-lib\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.912917 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.912829    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0555c0a2-fbef-4218-8f46-7beab92fb196-ovnkube-config\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.914175 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.914159    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0555c0a2-fbef-4218-8f46-7beab92fb196-ovn-node-metrics-cert\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.924396 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.924372    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q68w\" (UniqueName: \"kubernetes.io/projected/0555c0a2-fbef-4218-8f46-7beab92fb196-kube-api-access-8q68w\") pod \"ovnkube-node-fc2l7\" (UID: \"0555c0a2-fbef-4218-8f46-7beab92fb196\") " pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:20.998479 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:20.998458    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:21.034046 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:21:21.034025    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0555c0a2_fbef_4218_8f46_7beab92fb196.slice/crio-d7e87ac24ead3d465e3ab6659c241f611b4ef5fe3f2a98daf238d6013c69558f WatchSource:0}: Error finding container d7e87ac24ead3d465e3ab6659c241f611b4ef5fe3f2a98daf238d6013c69558f: Status 404 returned error can't find the container with id d7e87ac24ead3d465e3ab6659c241f611b4ef5fe3f2a98daf238d6013c69558f
Sep 10 13:21:21.129216 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.129187    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7" event={"ID":"0555c0a2-fbef-4218-8f46-7beab92fb196","Type":"ContainerStarted","Data":"d7e87ac24ead3d465e3ab6659c241f611b4ef5fe3f2a98daf238d6013c69558f"}
Sep 10 13:21:21.130610 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.130592    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/ovnkube-controller/0.log"
Sep 10 13:21:21.131897 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.131882    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/kube-rbac-proxy-ovn-metrics/0.log"
Sep 10 13:21:21.132256 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.132242    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/kube-rbac-proxy-node/0.log"
Sep 10 13:21:21.132564 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.132549    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/ovn-acl-logging/0.log"
Sep 10 13:21:21.132971 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.132958    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h6v4_28f104e3-2da9-4ad6-bb98-361fd93879cd/ovn-controller/0.log"
Sep 10 13:21:21.133016 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.133007    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4" event={"ID":"28f104e3-2da9-4ad6-bb98-361fd93879cd","Type":"ContainerDied","Data":"2a5018238dfbc30e5296271323b6ab2540fae4f793143c7a24b9540da59ffdf7"}
Sep 10 13:21:21.133072 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.133062    2551 scope.go:117] "RemoveContainer" containerID="a6541cb197e0c73123a504cf16c1612f1a52d9c62efd2dbbede843b1fd506033"
Sep 10 13:21:21.133105 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.133070    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7h6v4"
Sep 10 13:21:21.140554 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.140539    2551 scope.go:117] "RemoveContainer" containerID="e0cd66c96222ce4c8d7ad13433edba2a1055709a4561d002e09d15c2b9c2fcc7"
Sep 10 13:21:21.155782 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.155763    2551 scope.go:117] "RemoveContainer" containerID="99bc21f10cffe884b344398c0914c80662b2ebb5fcf7b4eceae32ca2d226ada4"
Sep 10 13:21:21.161295 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.161276    2551 scope.go:117] "RemoveContainer" containerID="0fed014842c901cabe7de92fbe77c71f7220e2b15617bfa6563fbd4f21c4f6e1"
Sep 10 13:21:21.166770 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.166754    2551 scope.go:117] "RemoveContainer" containerID="c065e04bf85a07432cd35109e029e45de1c70dc318ffd4bb074061368a7af67f"
Sep 10 13:21:21.168301 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.168279    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-7h6v4"]
Sep 10 13:21:21.172165 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.172151    2551 scope.go:117] "RemoveContainer" containerID="765d2b4e3fe4367f64549fd1aef0b2699661d6311e16a630d995edc907d2b7ec"
Sep 10 13:21:21.174877 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.174861    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-7h6v4"]
Sep 10 13:21:21.178001 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.177986    2551 scope.go:117] "RemoveContainer" containerID="fafafa2b0be7011e5befc42d412c9c73a373855c4ac8340d1ac200cf774cf526"
Sep 10 13:21:21.183307 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.183291    2551 scope.go:117] "RemoveContainer" containerID="dbfd55d99a97f95b80cdf3bcd468a54060e39fdf5bc7ee97053efd00135a76c0"
Sep 10 13:21:21.707267 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:21.707207    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28f104e3-2da9-4ad6-bb98-361fd93879cd" path="/var/lib/kubelet/pods/28f104e3-2da9-4ad6-bb98-361fd93879cd/volumes"
Sep 10 13:21:22.138864 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:22.138829    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7" event={"ID":"0555c0a2-fbef-4218-8f46-7beab92fb196","Type":"ContainerStarted","Data":"219689769eff918d912423421de15e0d6a1b44a0b49fffe5842a737addb1e9b4"}
Sep 10 13:21:22.139258 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:22.138899    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7" event={"ID":"0555c0a2-fbef-4218-8f46-7beab92fb196","Type":"ContainerStarted","Data":"56e9381dd84a798be908f9381d62d210372f9db30ebdb8576b0a5ec29b79f6b6"}
Sep 10 13:21:22.139258 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:22.138939    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7" event={"ID":"0555c0a2-fbef-4218-8f46-7beab92fb196","Type":"ContainerStarted","Data":"8880e9f7f000a22e34567ee1415a6bd2bd9accbb22e23c7fae89a891b9fd362f"}
Sep 10 13:21:22.139258 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:22.138967    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7" event={"ID":"0555c0a2-fbef-4218-8f46-7beab92fb196","Type":"ContainerStarted","Data":"11741cf7a13abf053bed0d25aebf4797496b1abf128a233c0b5e55caf4bda583"}
Sep 10 13:21:22.139258 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:22.138993    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7" event={"ID":"0555c0a2-fbef-4218-8f46-7beab92fb196","Type":"ContainerStarted","Data":"81aea42655bb4c195583103afc861acf1ae32fff77bcef3d853c4bd8d71ddd7f"}
Sep 10 13:21:22.139258 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:22.139027    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7" event={"ID":"0555c0a2-fbef-4218-8f46-7beab92fb196","Type":"ContainerStarted","Data":"46f38e98e03ae6eb6d88d27a5af2ae56dfedf38ea973fcb11ae7086a623e978a"}
Sep 10 13:21:22.703758 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:22.703728    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:22.703919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:22.703731    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:22.703919 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:22.703834    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:22.703919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:22.703848    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:22.704076 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:22.703936    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:22.704076 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:22.703988    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:24.146774 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:24.146746    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7" event={"ID":"0555c0a2-fbef-4218-8f46-7beab92fb196","Type":"ContainerStarted","Data":"fbabbcfaad61ffeafab432660b5e1ed316607948bf82a2da363a4b7f836c3eee"}
Sep 10 13:21:24.676158 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:24.676129    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:21:24.704412 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:24.704387    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:24.704493 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:24.704389    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:24.704618 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:24.704598    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:24.704686 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:24.704389    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:24.704726 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:24.704483    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:24.704726 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:24.704701    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:26.154696 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:26.154629    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7" event={"ID":"0555c0a2-fbef-4218-8f46-7beab92fb196","Type":"ContainerStarted","Data":"33ddc673710613cc0e3167292115b5fa4b0e80d776ef85ef8b5eb7d857437a5b"}
Sep 10 13:21:26.155005 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:26.154883    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:26.169090 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:26.169065    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:26.184738 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:26.184704    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7" podStartSLOduration=6.1846928 podStartE2EDuration="6.1846928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:21:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:26.184313923 +0000 UTC m=+197.003352322" watchObservedRunningTime="2026-09-10 13:21:26.1846928 +0000 UTC m=+197.003731200"
Sep 10 13:21:26.704243 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:26.704205    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:26.704243 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:26.704237    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:26.704457 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:26.704324    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:26.704457 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:26.704383    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:26.704520 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:26.704463    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:26.704612 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:26.704568    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:27.157347 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:27.157323    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:27.157810 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:27.157388    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:27.171439 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:27.171416    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:28.704220 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:28.704193    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:28.704616 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:28.704194    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:28.704616 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:28.704279    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:28.704616 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:28.704194    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:28.704616 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:28.704357    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:28.704616 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:28.704443    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:29.676608 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:29.676567    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:21:30.165508 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:30.165483    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:21:30.165863 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:30.165518    2551 generic.go:356] "Generic (PLEG): container finished" podID="5ec389cc-200a-4541-a957-8f419ee31f55" containerID="4da708d66d054e066240350f5a24e6f9c136519030a06c8fdccc51d55a471c05" exitCode=1
Sep 10 13:21:30.165863 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:30.165551    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mtb95" event={"ID":"5ec389cc-200a-4541-a957-8f419ee31f55","Type":"ContainerDied","Data":"4da708d66d054e066240350f5a24e6f9c136519030a06c8fdccc51d55a471c05"}
Sep 10 13:21:30.165936 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:30.165868    2551 scope.go:117] "RemoveContainer" containerID="4da708d66d054e066240350f5a24e6f9c136519030a06c8fdccc51d55a471c05"
Sep 10 13:21:30.703937 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:30.703861    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:30.704057 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:30.703965    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:30.704057 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:30.703979    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:30.704057 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:30.703993    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:30.704158 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:30.704074    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:30.704158 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:30.704145    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:31.075846 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:31.075813    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:31.075990 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:31.075963    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:21:31.076030 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:31.076025    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs podName:8bc0fa69-2ca1-4cd6-b004-88863086d37d nodeName:}" failed. No retries permitted until 2026-09-10 13:22:35.076010702 +0000 UTC m=+265.895049100 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs") pod "network-metrics-daemon-xxmwh" (UID: "8bc0fa69-2ca1-4cd6-b004-88863086d37d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:21:31.170246 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:31.170224    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:21:31.170600 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:31.170273    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mtb95" event={"ID":"5ec389cc-200a-4541-a957-8f419ee31f55","Type":"ContainerStarted","Data":"f24718ba56792f0a0eba4b4b11252cf9e4f0bc0ddb9f76346f9e2bed0ea86ab3"}
Sep 10 13:21:31.176744 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:31.176724    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrm9s\" (UniqueName: \"kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s\") pod \"network-check-target-82cgh\" (UID: \"4c893996-bc6b-46f0-ab3f-b98dcb115908\") " pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:31.176860 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:31.176848    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:21:31.176899 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:31.176863    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:21:31.176899 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:31.176872    2551 projected.go:196] Error preparing data for projected volume kube-api-access-xrm9s for pod openshift-network-diagnostics/network-check-target-82cgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:21:31.176981 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:31.176912    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s podName:4c893996-bc6b-46f0-ab3f-b98dcb115908 nodeName:}" failed. No retries permitted until 2026-09-10 13:22:35.176901518 +0000 UTC m=+265.995939896 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrm9s" (UniqueName: "kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s") pod "network-check-target-82cgh" (UID: "4c893996-bc6b-46f0-ab3f-b98dcb115908") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:21:31.277491 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:31.277467    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:31.277618 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:31.277606    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:21:31.277689 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:31.277675    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret podName:68de88a2-c141-4c07-9bfd-9ad2aa25b835 nodeName:}" failed. No retries permitted until 2026-09-10 13:22:35.277658375 +0000 UTC m=+266.096696763 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret") pod "global-pull-secret-syncer-fnfwb" (UID: "68de88a2-c141-4c07-9bfd-9ad2aa25b835") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:21:32.703616 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:32.703583    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:32.704074 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:32.703588    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:32.704074 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:32.703685    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:32.704074 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:32.703784    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:32.704074 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:32.703587    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:32.704074 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:32.703891    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:34.677778 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:34.677738    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:21:34.704311 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:34.704287    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:34.704411 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:34.704309    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:34.704411 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:34.704309    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:34.704411 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:34.704379    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:34.704525 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:34.704438    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:34.704563 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:34.704529    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:36.704117 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:36.704086    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:36.704117 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:36.704121    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:36.704623 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:36.704096    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:36.704623 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:36.704188    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:36.704623 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:36.704309    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:36.704623 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:36.704343    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:38.704412 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:38.704374    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:38.704829 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:38.704384    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:38.704829 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:38.704482    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:38.704829 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:38.704384    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:38.704829 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:38.704614    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:38.704829 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:38.704700    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:39.679062 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:39.679030    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:21:40.703785 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:40.703745    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:40.704152 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:40.703867    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:40.704152 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:40.703873    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:40.704152 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:40.703896    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:40.704152 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:40.703976    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:40.704152 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:40.704049    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:42.704088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:42.704047    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:42.704088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:42.704047    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:42.704088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:42.704066    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:42.704520 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:42.704285    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:42.704520 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:42.704414    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:42.704520 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:42.704445    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:44.679797 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:44.679763    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:21:44.704031 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:44.704011    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:44.704110 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:44.704011    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:44.704146 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:44.704108    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:44.704212 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:44.704196    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:44.704249 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:44.704011    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:44.704282 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:44.704265    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:46.704394 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:46.704359    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:46.704790 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:46.704402    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:46.704790 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:46.704450    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:46.704790 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:46.704463    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:46.704790 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:46.704519    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:46.704790 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:46.704594    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:47.384907 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:47.384718    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xxmwh"]
Sep 10 13:21:47.385024 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:47.384937    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:47.385060 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:47.385021    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:47.440798 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:47.440760    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fnfwb"]
Sep 10 13:21:47.440988 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:47.440864    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:47.440988 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:47.440972    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:47.441509 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:47.441479    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-82cgh"]
Sep 10 13:21:47.441637 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:47.441551    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:47.441698 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:47.441636    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:49.680843 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:49.680766    2551 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Sep 10 13:21:49.704993 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:49.704963    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:49.705116 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:49.705077    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:49.705183 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:49.705159    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:49.705263 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:49.705250    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:49.705304 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:49.705270    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:49.705352 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:49.705314    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:51.704400 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:51.704357    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:51.704898 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:51.704357    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:51.704898 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:51.704470    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:51.704898 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:51.704539    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:51.704898 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:51.704379    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:51.704898 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:51.704633    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:53.703870 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:53.703835    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:53.704294 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:53.703833    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:53.704294 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:53.703970    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82cgh" podUID="4c893996-bc6b-46f0-ab3f-b98dcb115908"
Sep 10 13:21:53.704294 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:53.703833    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:53.704294 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:53.704013    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xxmwh" podUID="8bc0fa69-2ca1-4cd6-b004-88863086d37d"
Sep 10 13:21:53.704294 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:21:53.704114    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fnfwb" podUID="68de88a2-c141-4c07-9bfd-9ad2aa25b835"
Sep 10 13:21:55.704708 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:55.704676    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:21:55.704708 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:55.704690    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:21:55.705182 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:55.704911    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:21:55.713636 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:55.713616    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 13:21:55.714802 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:55.714787    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 13:21:55.715333 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:55.715310    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 13:21:55.715413 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:55.715351    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9hn4m\""
Sep 10 13:21:55.715413 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:55.715381    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 13:21:55.715521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:55.715461    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4sjdq\""
Sep 10 13:21:55.715521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:55.715472    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-67hjc\""
Sep 10 13:21:56.233970 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.233925    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hlkvn"]
Sep 10 13:21:56.236302 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.236281    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.245000 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.244981    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 13:21:56.245087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.245008    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 13:21:56.245087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.245059    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vt7d7\""
Sep 10 13:21:56.245268 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.245256    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 13:21:56.245406 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.245391    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 13:21:56.247533 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.247521    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 13:21:56.255750 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.255738    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 13:21:56.331143 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.331116    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-accelerators-collector-config\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.331271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.331149    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/21c4a985-fa27-4fcb-bd5c-660cd4906719-root\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.331271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.331185    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.331271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.331237    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21c4a985-fa27-4fcb-bd5c-660cd4906719-sys\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.331380 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.331278    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-textfile\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.331380 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.331296    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-wtmp\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.331380 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.331311    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-tls\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.331380 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.331349    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21c4a985-fa27-4fcb-bd5c-660cd4906719-metrics-client-ca\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.331380 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.331373    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znxw4\" (UniqueName: \"kubernetes.io/projected/21c4a985-fa27-4fcb-bd5c-660cd4906719-kube-api-access-znxw4\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.432428 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.432398    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.432428 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.432430    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21c4a985-fa27-4fcb-bd5c-660cd4906719-sys\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.432644 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.432450    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-textfile\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.432644 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.432491    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-wtmp\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.432644 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.432516    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-tls\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.432644 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.432531    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21c4a985-fa27-4fcb-bd5c-660cd4906719-sys\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.432644 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.432566    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21c4a985-fa27-4fcb-bd5c-660cd4906719-metrics-client-ca\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.432644 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.432611    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znxw4\" (UniqueName: \"kubernetes.io/projected/21c4a985-fa27-4fcb-bd5c-660cd4906719-kube-api-access-znxw4\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.432644 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.432638    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-wtmp\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.432970 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.432656    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-accelerators-collector-config\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.432970 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.432690    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/21c4a985-fa27-4fcb-bd5c-660cd4906719-root\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.432970 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.432749    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/21c4a985-fa27-4fcb-bd5c-660cd4906719-root\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.432970 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.432823    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-textfile\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.433194 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.433168    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-accelerators-collector-config\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.433308 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.433234    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21c4a985-fa27-4fcb-bd5c-660cd4906719-metrics-client-ca\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.435762 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.435745    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.435841 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.435824    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/21c4a985-fa27-4fcb-bd5c-660cd4906719-node-exporter-tls\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.448209 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.448185    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znxw4\" (UniqueName: \"kubernetes.io/projected/21c4a985-fa27-4fcb-bd5c-660cd4906719-kube-api-access-znxw4\") pod \"node-exporter-hlkvn\" (UID: \"21c4a985-fa27-4fcb-bd5c-660cd4906719\") " pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:56.545146 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:56.545078    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hlkvn"
Sep 10 13:21:57.229040 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:57.229006    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hlkvn" event={"ID":"21c4a985-fa27-4fcb-bd5c-660cd4906719","Type":"ContainerStarted","Data":"0ffc2a1eaf7c587c067200096edf041669a47c67f35cf890e085e63090066ce7"}
Sep 10 13:21:58.233297 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:58.233261    2551 generic.go:356] "Generic (PLEG): container finished" podID="21c4a985-fa27-4fcb-bd5c-660cd4906719" containerID="ced09d84d2b85b4ce25648b0852783b7ad74a961c80199bc40fe33feea50edec" exitCode=0
Sep 10 13:21:58.233297 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:58.233300    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hlkvn" event={"ID":"21c4a985-fa27-4fcb-bd5c-660cd4906719","Type":"ContainerDied","Data":"ced09d84d2b85b4ce25648b0852783b7ad74a961c80199bc40fe33feea50edec"}
Sep 10 13:21:59.171884 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.171856    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fc2l7"
Sep 10 13:21:59.220961 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.220937    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-117.ec2.internal" event="NodeReady"
Sep 10 13:21:59.237730 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.237706    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hlkvn" event={"ID":"21c4a985-fa27-4fcb-bd5c-660cd4906719","Type":"ContainerStarted","Data":"f2503ad2eedaaab71dd245f6ef589e4b97c852d3c88232854827af511f56c291"}
Sep 10 13:21:59.238006 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.237747    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hlkvn" event={"ID":"21c4a985-fa27-4fcb-bd5c-660cd4906719","Type":"ContainerStarted","Data":"0252671c6b5b89ba968ef94a3933e088523661f46fa4f9e696cb5f7694348321"}
Sep 10 13:21:59.297011 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.296966    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hlkvn" podStartSLOduration=2.149499479 podStartE2EDuration="3.296934516s" totalImagesPullingTime="1.087263508s" totalInitContainerRuntime="60.171529ms" isStatefulPod=true podCreationTimestamp="2026-09-10 13:21:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:59.296838393 +0000 UTC m=+230.115876792" watchObservedRunningTime="2026-09-10 13:21:59.296934516 +0000 UTC m=+230.115972915"
Sep 10 13:21:59.356760 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.356738    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-r7sq8"]
Sep 10 13:21:59.358357 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.358343    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r7sq8"
Sep 10 13:21:59.364631 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.364612    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nzsjz\""
Sep 10 13:21:59.365039 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.365010    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 13:21:59.365039 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.365025    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 13:21:59.365173 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.365064    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zszcl"]
Sep 10 13:21:59.365173 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.365021    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 13:21:59.366912 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.366898    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.374025 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.374006    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 13:21:59.374233 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.374219    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 13:21:59.374293 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.374281    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 13:21:59.374334 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.374323    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 13:21:59.374483 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.374471    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pbl4r\""
Sep 10 13:21:59.399446 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.399427    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r7sq8"]
Sep 10 13:21:59.404171 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.404149    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zszcl"]
Sep 10 13:21:59.428804 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.428755    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9whh7"]
Sep 10 13:21:59.430768 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.430751    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9whh7"
Sep 10 13:21:59.434082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.434067    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 13:21:59.434178 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.434112    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7k2sl\""
Sep 10 13:21:59.434316 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.434299    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 13:21:59.453755 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.453731    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/095d9660-3d0b-48eb-abf9-ddd7b60ec7b9-config-volume\") pod \"dns-default-9whh7\" (UID: \"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9\") " pod="openshift-dns/dns-default-9whh7"
Sep 10 13:21:59.453843 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.453764    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.453900 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.453845    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7dff7693-9534-488c-b000-5ddd0bb3abad-cert\") pod \"ingress-canary-r7sq8\" (UID: \"7dff7693-9534-488c-b000-5ddd0bb3abad\") " pod="openshift-ingress-canary/ingress-canary-r7sq8"
Sep 10 13:21:59.453900 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.453890    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-crio-socket\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.453982 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.453909    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-data-volume\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.453982 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.453930    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/095d9660-3d0b-48eb-abf9-ddd7b60ec7b9-metrics-tls\") pod \"dns-default-9whh7\" (UID: \"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9\") " pod="openshift-dns/dns-default-9whh7"
Sep 10 13:21:59.453982 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.453951    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtfp5\" (UniqueName: \"kubernetes.io/projected/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-kube-api-access-wtfp5\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.453982 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.453974    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.454144 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.454004    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/095d9660-3d0b-48eb-abf9-ddd7b60ec7b9-tmp-dir\") pod \"dns-default-9whh7\" (UID: \"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9\") " pod="openshift-dns/dns-default-9whh7"
Sep 10 13:21:59.454144 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.454018    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5j9c\" (UniqueName: \"kubernetes.io/projected/095d9660-3d0b-48eb-abf9-ddd7b60ec7b9-kube-api-access-h5j9c\") pod \"dns-default-9whh7\" (UID: \"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9\") " pod="openshift-dns/dns-default-9whh7"
Sep 10 13:21:59.454144 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.454037    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdrws\" (UniqueName: \"kubernetes.io/projected/7dff7693-9534-488c-b000-5ddd0bb3abad-kube-api-access-hdrws\") pod \"ingress-canary-r7sq8\" (UID: \"7dff7693-9534-488c-b000-5ddd0bb3abad\") " pod="openshift-ingress-canary/ingress-canary-r7sq8"
Sep 10 13:21:59.461709 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.461689    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9whh7"]
Sep 10 13:21:59.554682 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.554653    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7dff7693-9534-488c-b000-5ddd0bb3abad-cert\") pod \"ingress-canary-r7sq8\" (UID: \"7dff7693-9534-488c-b000-5ddd0bb3abad\") " pod="openshift-ingress-canary/ingress-canary-r7sq8"
Sep 10 13:21:59.554806 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.554705    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-crio-socket\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.554806 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.554729    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-data-volume\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.554806 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.554756    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/095d9660-3d0b-48eb-abf9-ddd7b60ec7b9-metrics-tls\") pod \"dns-default-9whh7\" (UID: \"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9\") " pod="openshift-dns/dns-default-9whh7"
Sep 10 13:21:59.554806 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.554792    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtfp5\" (UniqueName: \"kubernetes.io/projected/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-kube-api-access-wtfp5\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.555021 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.554829    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.555021 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.554838    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-crio-socket\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.555021 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.554859    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/095d9660-3d0b-48eb-abf9-ddd7b60ec7b9-tmp-dir\") pod \"dns-default-9whh7\" (UID: \"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9\") " pod="openshift-dns/dns-default-9whh7"
Sep 10 13:21:59.555021 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.554883    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5j9c\" (UniqueName: \"kubernetes.io/projected/095d9660-3d0b-48eb-abf9-ddd7b60ec7b9-kube-api-access-h5j9c\") pod \"dns-default-9whh7\" (UID: \"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9\") " pod="openshift-dns/dns-default-9whh7"
Sep 10 13:21:59.555021 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.554920    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdrws\" (UniqueName: \"kubernetes.io/projected/7dff7693-9534-488c-b000-5ddd0bb3abad-kube-api-access-hdrws\") pod \"ingress-canary-r7sq8\" (UID: \"7dff7693-9534-488c-b000-5ddd0bb3abad\") " pod="openshift-ingress-canary/ingress-canary-r7sq8"
Sep 10 13:21:59.555244 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.555077    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/095d9660-3d0b-48eb-abf9-ddd7b60ec7b9-config-volume\") pod \"dns-default-9whh7\" (UID: \"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9\") " pod="openshift-dns/dns-default-9whh7"
Sep 10 13:21:59.555244 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.555077    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-data-volume\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.555244 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.555113    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.555433 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.555405    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.555433 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.555418    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/095d9660-3d0b-48eb-abf9-ddd7b60ec7b9-tmp-dir\") pod \"dns-default-9whh7\" (UID: \"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9\") " pod="openshift-dns/dns-default-9whh7"
Sep 10 13:21:59.555696 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.555677    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/095d9660-3d0b-48eb-abf9-ddd7b60ec7b9-config-volume\") pod \"dns-default-9whh7\" (UID: \"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9\") " pod="openshift-dns/dns-default-9whh7"
Sep 10 13:21:59.557314 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.557292    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.557487 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.557472    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/095d9660-3d0b-48eb-abf9-ddd7b60ec7b9-metrics-tls\") pod \"dns-default-9whh7\" (UID: \"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9\") " pod="openshift-dns/dns-default-9whh7"
Sep 10 13:21:59.557589 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.557562    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7dff7693-9534-488c-b000-5ddd0bb3abad-cert\") pod \"ingress-canary-r7sq8\" (UID: \"7dff7693-9534-488c-b000-5ddd0bb3abad\") " pod="openshift-ingress-canary/ingress-canary-r7sq8"
Sep 10 13:21:59.565539 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.565520    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5j9c\" (UniqueName: \"kubernetes.io/projected/095d9660-3d0b-48eb-abf9-ddd7b60ec7b9-kube-api-access-h5j9c\") pod \"dns-default-9whh7\" (UID: \"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9\") " pod="openshift-dns/dns-default-9whh7"
Sep 10 13:21:59.566153 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.566130    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtfp5\" (UniqueName: \"kubernetes.io/projected/e5ea87df-d83a-4e39-9719-41b92cdfbd3a-kube-api-access-wtfp5\") pod \"insights-runtime-extractor-zszcl\" (UID: \"e5ea87df-d83a-4e39-9719-41b92cdfbd3a\") " pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.567762 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.567743    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdrws\" (UniqueName: \"kubernetes.io/projected/7dff7693-9534-488c-b000-5ddd0bb3abad-kube-api-access-hdrws\") pod \"ingress-canary-r7sq8\" (UID: \"7dff7693-9534-488c-b000-5ddd0bb3abad\") " pod="openshift-ingress-canary/ingress-canary-r7sq8"
Sep 10 13:21:59.666756 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.666728    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r7sq8"
Sep 10 13:21:59.674185 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.674161    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zszcl"
Sep 10 13:21:59.739829 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.739733    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9whh7"
Sep 10 13:21:59.984798 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:21:59.984770    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zszcl"]
Sep 10 13:21:59.988045 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:21:59.988020    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5ea87df_d83a_4e39_9719_41b92cdfbd3a.slice/crio-f6763b8810fa29b4440ec36fd20692d2164bc0925a5cfec25fc4f88998c465cb WatchSource:0}: Error finding container f6763b8810fa29b4440ec36fd20692d2164bc0925a5cfec25fc4f88998c465cb: Status 404 returned error can't find the container with id f6763b8810fa29b4440ec36fd20692d2164bc0925a5cfec25fc4f88998c465cb
Sep 10 13:22:00.008069 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.008040    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r7sq8"]
Sep 10 13:22:00.012524 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:22:00.012497    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dff7693_9534_488c_b000_5ddd0bb3abad.slice/crio-c147a40f496530bf48c1419ab3e589fcbd7ae311651c3095ed452d5034a958f2 WatchSource:0}: Error finding container c147a40f496530bf48c1419ab3e589fcbd7ae311651c3095ed452d5034a958f2: Status 404 returned error can't find the container with id c147a40f496530bf48c1419ab3e589fcbd7ae311651c3095ed452d5034a958f2
Sep 10 13:22:00.056012 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.055989    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9whh7"]
Sep 10 13:22:00.060121 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:22:00.060092    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod095d9660_3d0b_48eb_abf9_ddd7b60ec7b9.slice/crio-5f783674887fe6087082d97d5facf9a0abd9012b5d76bcdbf981fcf6ff9fa696 WatchSource:0}: Error finding container 5f783674887fe6087082d97d5facf9a0abd9012b5d76bcdbf981fcf6ff9fa696: Status 404 returned error can't find the container with id 5f783674887fe6087082d97d5facf9a0abd9012b5d76bcdbf981fcf6ff9fa696
Sep 10 13:22:00.201231 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.201202    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6bddc647f4-scbfg"]
Sep 10 13:22:00.203639 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.203622    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.206992 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.206966    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 10 13:22:00.207108 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.206966    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 10 13:22:00.207108 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.207006    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 10 13:22:00.210376 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.210359    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-4qf83uv7chghu\""
Sep 10 13:22:00.210904 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.210883    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-69kht\""
Sep 10 13:22:00.210993 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.210908    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 10 13:22:00.210993 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.210939    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 10 13:22:00.241907 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.241855    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zszcl" event={"ID":"e5ea87df-d83a-4e39-9719-41b92cdfbd3a","Type":"ContainerStarted","Data":"e5bc441039c66d6fb32d85bd3cd4651db065d4f7846444808a7dc3d507f7cfe1"}
Sep 10 13:22:00.242188 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.241905    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zszcl" event={"ID":"e5ea87df-d83a-4e39-9719-41b92cdfbd3a","Type":"ContainerStarted","Data":"f6763b8810fa29b4440ec36fd20692d2164bc0925a5cfec25fc4f88998c465cb"}
Sep 10 13:22:00.243091 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.243061    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9whh7" event={"ID":"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9","Type":"ContainerStarted","Data":"5f783674887fe6087082d97d5facf9a0abd9012b5d76bcdbf981fcf6ff9fa696"}
Sep 10 13:22:00.243885 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.243864    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6bddc647f4-scbfg"]
Sep 10 13:22:00.244409 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.244383    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r7sq8" event={"ID":"7dff7693-9534-488c-b000-5ddd0bb3abad","Type":"ContainerStarted","Data":"c147a40f496530bf48c1419ab3e589fcbd7ae311651c3095ed452d5034a958f2"}
Sep 10 13:22:00.261070 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.261051    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.261172 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.261118    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.261230 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.261166    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nltcm\" (UniqueName: \"kubernetes.io/projected/8a6461b6-32a7-4faa-8beb-69c46e3324a9-kube-api-access-nltcm\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.261397 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.261382    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a6461b6-32a7-4faa-8beb-69c46e3324a9-metrics-client-ca\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.261459 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.261444    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.261505 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.261483    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.261538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.261513    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-tls\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.261638 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.261626    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-grpc-tls\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.362273 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.362246    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-grpc-tls\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.362358 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.362284    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.362444 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.362424    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.362481 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.362456    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nltcm\" (UniqueName: \"kubernetes.io/projected/8a6461b6-32a7-4faa-8beb-69c46e3324a9-kube-api-access-nltcm\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.362481 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.362475    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a6461b6-32a7-4faa-8beb-69c46e3324a9-metrics-client-ca\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.362764 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.362635    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.362764 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.362691    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.362764 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.362725    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-tls\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.363454 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.363397    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a6461b6-32a7-4faa-8beb-69c46e3324a9-metrics-client-ca\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.365002 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.364957    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.365182 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.365163    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.365182 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.365174    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-grpc-tls\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.365315 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.365254    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.365315 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.365254    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.365377 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.365315    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8a6461b6-32a7-4faa-8beb-69c46e3324a9-secret-thanos-querier-tls\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.380264 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.380247    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nltcm\" (UniqueName: \"kubernetes.io/projected/8a6461b6-32a7-4faa-8beb-69c46e3324a9-kube-api-access-nltcm\") pod \"thanos-querier-6bddc647f4-scbfg\" (UID: \"8a6461b6-32a7-4faa-8beb-69c46e3324a9\") " pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.512134 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.512080    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:00.759419 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.759226    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx"]
Sep 10 13:22:00.761395 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.761370    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx"
Sep 10 13:22:00.765952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.765896    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xgwsq\""
Sep 10 13:22:00.766729 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.766711    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 13:22:00.784021 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.783967    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx"]
Sep 10 13:22:00.822539 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.822479    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6bddc647f4-scbfg"]
Sep 10 13:22:00.841810 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.841781    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-8479b9f7c4-68qk5"]
Sep 10 13:22:00.844163 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.844141    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.851810 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.851643    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 10 13:22:00.851941 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.851929    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-b3ss6hnrjrhsv\""
Sep 10 13:22:00.852426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.852407    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-krgqd\""
Sep 10 13:22:00.852638 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.852621    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 10 13:22:00.852950 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.852840    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 10 13:22:00.853091 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.853076    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 13:22:00.866495 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.866470    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f4aed292-e888-4f34-86af-29d1e02d5b90-secret-metrics-server-tls\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.866731 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.866709    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f4aed292-e888-4f34-86af-29d1e02d5b90-metrics-server-audit-profiles\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.866861 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.866757    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f4aed292-e888-4f34-86af-29d1e02d5b90-client-ca-bundle\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.866861 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.866784    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f4aed292-e888-4f34-86af-29d1e02d5b90-audit-log\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.866861 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.866856    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvvqn\" (UniqueName: \"kubernetes.io/projected/c0b64459-3f7a-4e4d-9225-c9fe33220957-kube-api-access-bvvqn\") pod \"monitoring-plugin-85bbbbbf4-xnknx\" (UID: \"c0b64459-3f7a-4e4d-9225-c9fe33220957\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx"
Sep 10 13:22:00.867112 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.866891    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqtsf\" (UniqueName: \"kubernetes.io/projected/f4aed292-e888-4f34-86af-29d1e02d5b90-kube-api-access-lqtsf\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.867112 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.866915    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0b64459-3f7a-4e4d-9225-c9fe33220957-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-xnknx\" (UID: \"c0b64459-3f7a-4e4d-9225-c9fe33220957\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx"
Sep 10 13:22:00.867112 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.866957    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4aed292-e888-4f34-86af-29d1e02d5b90-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.867112 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.866990    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f4aed292-e888-4f34-86af-29d1e02d5b90-secret-metrics-server-client-certs\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.869328 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.868388    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8479b9f7c4-68qk5"]
Sep 10 13:22:00.969191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.968112    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvvqn\" (UniqueName: \"kubernetes.io/projected/c0b64459-3f7a-4e4d-9225-c9fe33220957-kube-api-access-bvvqn\") pod \"monitoring-plugin-85bbbbbf4-xnknx\" (UID: \"c0b64459-3f7a-4e4d-9225-c9fe33220957\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx"
Sep 10 13:22:00.969191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.968174    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqtsf\" (UniqueName: \"kubernetes.io/projected/f4aed292-e888-4f34-86af-29d1e02d5b90-kube-api-access-lqtsf\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.969191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.968215    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0b64459-3f7a-4e4d-9225-c9fe33220957-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-xnknx\" (UID: \"c0b64459-3f7a-4e4d-9225-c9fe33220957\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx"
Sep 10 13:22:00.969191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.968253    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4aed292-e888-4f34-86af-29d1e02d5b90-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.969191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.968282    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f4aed292-e888-4f34-86af-29d1e02d5b90-secret-metrics-server-client-certs\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.969191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.968336    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f4aed292-e888-4f34-86af-29d1e02d5b90-secret-metrics-server-tls\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.969191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.968373    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f4aed292-e888-4f34-86af-29d1e02d5b90-metrics-server-audit-profiles\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.969191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.968399    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f4aed292-e888-4f34-86af-29d1e02d5b90-client-ca-bundle\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.969191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.968426    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f4aed292-e888-4f34-86af-29d1e02d5b90-audit-log\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.969191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.968859    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f4aed292-e888-4f34-86af-29d1e02d5b90-audit-log\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.970279 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.969856    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f4aed292-e888-4f34-86af-29d1e02d5b90-metrics-server-audit-profiles\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.970700 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.970658    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4aed292-e888-4f34-86af-29d1e02d5b90-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.974064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.974024    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f4aed292-e888-4f34-86af-29d1e02d5b90-client-ca-bundle\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.975425 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.974912    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f4aed292-e888-4f34-86af-29d1e02d5b90-secret-metrics-server-client-certs\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.975425 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.975371    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0b64459-3f7a-4e4d-9225-c9fe33220957-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-xnknx\" (UID: \"c0b64459-3f7a-4e4d-9225-c9fe33220957\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx"
Sep 10 13:22:00.977131 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.977110    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f4aed292-e888-4f34-86af-29d1e02d5b90-secret-metrics-server-tls\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:00.981321 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.981299    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvvqn\" (UniqueName: \"kubernetes.io/projected/c0b64459-3f7a-4e4d-9225-c9fe33220957-kube-api-access-bvvqn\") pod \"monitoring-plugin-85bbbbbf4-xnknx\" (UID: \"c0b64459-3f7a-4e4d-9225-c9fe33220957\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx"
Sep 10 13:22:00.984744 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:00.984705    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqtsf\" (UniqueName: \"kubernetes.io/projected/f4aed292-e888-4f34-86af-29d1e02d5b90-kube-api-access-lqtsf\") pod \"metrics-server-8479b9f7c4-68qk5\" (UID: \"f4aed292-e888-4f34-86af-29d1e02d5b90\") " pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:01.072062 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:01.071460    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx"
Sep 10 13:22:01.157288 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:01.157223    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:01.251030 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:01.250987    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zszcl" event={"ID":"e5ea87df-d83a-4e39-9719-41b92cdfbd3a","Type":"ContainerStarted","Data":"897ad1c6e2b001f4146dc78b625d03ba6c166229b8eaa1e295242b60b30c6c56"}
Sep 10 13:22:01.252890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:01.252865    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg" event={"ID":"8a6461b6-32a7-4faa-8beb-69c46e3324a9","Type":"ContainerStarted","Data":"dc4e6002b6f38ffc182fe8b7b7d781b990724e5661fda111a0eb4646adbf3b65"}
Sep 10 13:22:01.382630 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:01.382569    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx"]
Sep 10 13:22:01.384768 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:22:01.384742    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0b64459_3f7a_4e4d_9225_c9fe33220957.slice/crio-f82416a4d92834b0bf1410d44b9edc7aa8a50c074a899e6d14ba5c907df06d09 WatchSource:0}: Error finding container f82416a4d92834b0bf1410d44b9edc7aa8a50c074a899e6d14ba5c907df06d09: Status 404 returned error can't find the container with id f82416a4d92834b0bf1410d44b9edc7aa8a50c074a899e6d14ba5c907df06d09
Sep 10 13:22:01.446028 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:01.445991    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8479b9f7c4-68qk5"]
Sep 10 13:22:01.449272 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:22:01.449233    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4aed292_e888_4f34_86af_29d1e02d5b90.slice/crio-275be10cef25ae92e1ad62648ef1b6b43521bda731d6b05179bd8759ce14604c WatchSource:0}: Error finding container 275be10cef25ae92e1ad62648ef1b6b43521bda731d6b05179bd8759ce14604c: Status 404 returned error can't find the container with id 275be10cef25ae92e1ad62648ef1b6b43521bda731d6b05179bd8759ce14604c
Sep 10 13:22:02.256644 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.256601    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5" event={"ID":"f4aed292-e888-4f34-86af-29d1e02d5b90","Type":"ContainerStarted","Data":"275be10cef25ae92e1ad62648ef1b6b43521bda731d6b05179bd8759ce14604c"}
Sep 10 13:22:02.257831 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.257796    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx" event={"ID":"c0b64459-3f7a-4e4d-9225-c9fe33220957","Type":"ContainerStarted","Data":"f82416a4d92834b0bf1410d44b9edc7aa8a50c074a899e6d14ba5c907df06d09"}
Sep 10 13:22:02.895903 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.895857    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:22:02.898930 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.898903    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.906423 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.906397    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 13:22:02.906688 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.906670    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 13:22:02.907316 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.907286    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 13:22:02.907819 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.907764    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 13:22:02.908927 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.908048    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 13:22:02.908927 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.908237    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 13:22:02.908927 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.908445    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 13:22:02.908927 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.908792    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 13:22:02.909431 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.909026    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ef86qorjuqku3\""
Sep 10 13:22:02.909431 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.909248    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 13:22:02.909431 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.909331    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 13:22:02.909431 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.909428    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rjxd5\""
Sep 10 13:22:02.913405 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.913278    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 13:22:02.916542 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.916521    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 13:22:02.922273 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.922104    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 13:22:02.928946 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.928920    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 13:22:02.964244 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.964221    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:22:02.986192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986165    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986306 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986216    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986306 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986238    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986306 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986265    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b106a48-315d-4133-adb7-f8debe6f0e7c-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986306 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986303    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2qv8\" (UniqueName: \"kubernetes.io/projected/8b106a48-315d-4133-adb7-f8debe6f0e7c-kube-api-access-d2qv8\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986324    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986353    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-config\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986379    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986411    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986454    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986495    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986524    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986969 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986547    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986969 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986569    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986969 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986643    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986969 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986692    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b106a48-315d-4133-adb7-f8debe6f0e7c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986969 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986715    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986969 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986733    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986969 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986749    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:02.986969 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:02.986783    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.087476 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087436    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-config\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.087671 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087482    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.087671 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087514    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.087671 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087532    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.087671 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087563    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.087671 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087606    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.087671 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087624    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.087671 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087647    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.087671 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087677    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.088088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087698    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b106a48-315d-4133-adb7-f8debe6f0e7c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.088088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087725    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.088088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087748    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.088088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087766    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.088088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087791    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.088088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087825    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.088088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087850    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.088088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087883    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.088088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087908    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b106a48-315d-4133-adb7-f8debe6f0e7c-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.088088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087940    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2qv8\" (UniqueName: \"kubernetes.io/projected/8b106a48-315d-4133-adb7-f8debe6f0e7c-kube-api-access-d2qv8\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.088088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.087968    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.088608 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.088389    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.089144 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.089113    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.090954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.090926    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.091381 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.091356    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.091651 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.091615    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.091731 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.091646    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.091928 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.091907    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.092064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.091953    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.092395 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.092367    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.092591 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.092549    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.093137 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.092852    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.093296 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.093055    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.093457 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.093178    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b106a48-315d-4133-adb7-f8debe6f0e7c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.093739 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.093699    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b106a48-315d-4133-adb7-f8debe6f0e7c-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.094161 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.094117    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.094566 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.094543    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.094722 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.094679    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-config\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.094849 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.094824    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.094905 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.094830    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.121563 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.121509    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2qv8\" (UniqueName: \"kubernetes.io/projected/8b106a48-315d-4133-adb7-f8debe6f0e7c-kube-api-access-d2qv8\") pod \"prometheus-k8s-0\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:03.212655 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:03.212624    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:05.031206 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.031174    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:22:05.034385 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:22:05.034360    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b106a48_315d_4133_adb7_f8debe6f0e7c.slice/crio-90147faa30aedfe995a940f4fe239200356115a9243e65fd2fbe6d24931bbaad WatchSource:0}: Error finding container 90147faa30aedfe995a940f4fe239200356115a9243e65fd2fbe6d24931bbaad: Status 404 returned error can't find the container with id 90147faa30aedfe995a940f4fe239200356115a9243e65fd2fbe6d24931bbaad
Sep 10 13:22:05.267761 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.267723    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg" event={"ID":"8a6461b6-32a7-4faa-8beb-69c46e3324a9","Type":"ContainerStarted","Data":"274de489b211ef30ce35e46024893f3ed080b517be5337a580e9d4680fa029fe"}
Sep 10 13:22:05.267900 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.267806    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg" event={"ID":"8a6461b6-32a7-4faa-8beb-69c46e3324a9","Type":"ContainerStarted","Data":"27cd9548a56c07ad6ebdb052eab5d22fe195e09abafffab1601759cc96bb73a7"}
Sep 10 13:22:05.267900 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.267849    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg" event={"ID":"8a6461b6-32a7-4faa-8beb-69c46e3324a9","Type":"ContainerStarted","Data":"f4a3db558693e73912a3e09f8c2cf1a17221b16f90cf173148e4fd920a7c6cc1"}
Sep 10 13:22:05.269239 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.269215    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5" event={"ID":"f4aed292-e888-4f34-86af-29d1e02d5b90","Type":"ContainerStarted","Data":"2205ac4052fb126ddd5c3fb88eeb365b2f6787b82bb83bd609417d63d8229696"}
Sep 10 13:22:05.270883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.270856    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9whh7" event={"ID":"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9","Type":"ContainerStarted","Data":"5024a7bd821f339b7f6c0590c1003b3602035ce6de18eff2cc908ee7eafdb083"}
Sep 10 13:22:05.270978 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.270894    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9whh7" event={"ID":"095d9660-3d0b-48eb-abf9-ddd7b60ec7b9","Type":"ContainerStarted","Data":"fbafbc948f82090eb79eb086ffd3024d0365744ca6c3a8f16b37a654648b5dac"}
Sep 10 13:22:05.270978 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.270916    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9whh7"
Sep 10 13:22:05.272342 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.272320    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r7sq8" event={"ID":"7dff7693-9534-488c-b000-5ddd0bb3abad","Type":"ContainerStarted","Data":"e25e4b6863b1d251c3557e271b0cba8b4d37d0052cf770d9c896c0a2cfbbc741"}
Sep 10 13:22:05.273760 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.273742    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx" event={"ID":"c0b64459-3f7a-4e4d-9225-c9fe33220957","Type":"ContainerStarted","Data":"7d2e92b67292676d3d6c4caf81fcef156e500be1dff9f2e3073bba2643f420c1"}
Sep 10 13:22:05.273887 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.273867    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx"
Sep 10 13:22:05.275128 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.275100    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerStarted","Data":"90147faa30aedfe995a940f4fe239200356115a9243e65fd2fbe6d24931bbaad"}
Sep 10 13:22:05.277181 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.277164    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zszcl" event={"ID":"e5ea87df-d83a-4e39-9719-41b92cdfbd3a","Type":"ContainerStarted","Data":"958049c267127152465def5e76008d29f5ef3c9096e983069f38351f1b814330"}
Sep 10 13:22:05.278717 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.278699    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx"
Sep 10 13:22:05.310706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.310671    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5" podStartSLOduration=2.045230993 podStartE2EDuration="5.310659191s" totalImagesPullingTime="3.265428198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:22:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:05.310193626 +0000 UTC m=+236.129232027" watchObservedRunningTime="2026-09-10 13:22:05.310659191 +0000 UTC m=+236.129697591"
Sep 10 13:22:05.355085 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.355043    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zszcl" podStartSLOduration=1.7511292200000002 podStartE2EDuration="6.355031166s" totalImagesPullingTime="4.603901946s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:21:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:05.354597136 +0000 UTC m=+236.173635535" watchObservedRunningTime="2026-09-10 13:22:05.355031166 +0000 UTC m=+236.174069609"
Sep 10 13:22:05.430554 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.430509    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-r7sq8" podStartSLOduration=1.7342581799999999 podStartE2EDuration="6.430495828s" totalImagesPullingTime="4.696237648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:21:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:05.430267339 +0000 UTC m=+236.249305739" watchObservedRunningTime="2026-09-10 13:22:05.430495828 +0000 UTC m=+236.249534228"
Sep 10 13:22:05.480379 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.480329    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-xnknx" podStartSLOduration=2.147224572 podStartE2EDuration="5.480310133s" totalImagesPullingTime="3.333085561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:22:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:05.479168522 +0000 UTC m=+236.298206922" watchObservedRunningTime="2026-09-10 13:22:05.480310133 +0000 UTC m=+236.299348532"
Sep 10 13:22:05.531954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:05.531868    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9whh7" podStartSLOduration=1.883223821 podStartE2EDuration="6.531852144s" totalImagesPullingTime="4.648628323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:21:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:05.531529955 +0000 UTC m=+236.350568356" watchObservedRunningTime="2026-09-10 13:22:05.531852144 +0000 UTC m=+236.350890543"
Sep 10 13:22:06.416419 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:06.416385    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-ks28m"]
Sep 10 13:22:06.419455 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:06.419435    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-ks28m"
Sep 10 13:22:06.431350 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:06.431327    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 13:22:06.431472 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:06.431332    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-g9xxv\""
Sep 10 13:22:06.431647 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:06.431631    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 13:22:06.452840 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:06.452818    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-ks28m"]
Sep 10 13:22:06.516885 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:06.516865    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npljx\" (UniqueName: \"kubernetes.io/projected/75a92c91-c7a6-4b56-b4b1-7437ce6ccb85-kube-api-access-npljx\") pod \"downloads-85d98c998f-ks28m\" (UID: \"75a92c91-c7a6-4b56-b4b1-7437ce6ccb85\") " pod="openshift-console/downloads-85d98c998f-ks28m"
Sep 10 13:22:06.617935 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:06.617911    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npljx\" (UniqueName: \"kubernetes.io/projected/75a92c91-c7a6-4b56-b4b1-7437ce6ccb85-kube-api-access-npljx\") pod \"downloads-85d98c998f-ks28m\" (UID: \"75a92c91-c7a6-4b56-b4b1-7437ce6ccb85\") " pod="openshift-console/downloads-85d98c998f-ks28m"
Sep 10 13:22:06.629829 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:06.629807    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npljx\" (UniqueName: \"kubernetes.io/projected/75a92c91-c7a6-4b56-b4b1-7437ce6ccb85-kube-api-access-npljx\") pod \"downloads-85d98c998f-ks28m\" (UID: \"75a92c91-c7a6-4b56-b4b1-7437ce6ccb85\") " pod="openshift-console/downloads-85d98c998f-ks28m"
Sep 10 13:22:06.730006 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:06.729982    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-ks28m"
Sep 10 13:22:07.012417 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:07.012394    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-ks28m"]
Sep 10 13:22:07.014161 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:22:07.014129    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a92c91_c7a6_4b56_b4b1_7437ce6ccb85.slice/crio-df4ee0943b64a40b034599f27a6e35df44b37ec0bc94c2139d80cb5ef71256ac WatchSource:0}: Error finding container df4ee0943b64a40b034599f27a6e35df44b37ec0bc94c2139d80cb5ef71256ac: Status 404 returned error can't find the container with id df4ee0943b64a40b034599f27a6e35df44b37ec0bc94c2139d80cb5ef71256ac
Sep 10 13:22:07.286680 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:07.286598    2551 generic.go:356] "Generic (PLEG): container finished" podID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerID="50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e" exitCode=0
Sep 10 13:22:07.286835 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:07.286686    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerDied","Data":"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"}
Sep 10 13:22:07.287967 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:07.287915    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-ks28m" event={"ID":"75a92c91-c7a6-4b56-b4b1-7437ce6ccb85","Type":"ContainerStarted","Data":"df4ee0943b64a40b034599f27a6e35df44b37ec0bc94c2139d80cb5ef71256ac"}
Sep 10 13:22:07.290796 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:07.290772    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg" event={"ID":"8a6461b6-32a7-4faa-8beb-69c46e3324a9","Type":"ContainerStarted","Data":"e44ac26fb47b1be3805b90c8ab0f9f2891b2fc7cffac4be75df1c8ccc887e344"}
Sep 10 13:22:07.290890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:07.290836    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg" event={"ID":"8a6461b6-32a7-4faa-8beb-69c46e3324a9","Type":"ContainerStarted","Data":"7e275eff636af9c77f75a07aabf0bc96b5bbe7b562f83c22befb75fbb0195b1f"}
Sep 10 13:22:07.290890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:07.290878    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg" event={"ID":"8a6461b6-32a7-4faa-8beb-69c46e3324a9","Type":"ContainerStarted","Data":"40dab8aca4ed02888b3e52e056ecb98c342415da6fb54dfd26112e861854262d"}
Sep 10 13:22:08.294882 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:08.294847    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:09.305777 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:09.305692    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg"
Sep 10 13:22:09.356748 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:09.356692    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6bddc647f4-scbfg" podStartSLOduration=3.9350534 podStartE2EDuration="9.356672177s" totalImagesPullingTime="5.421618777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:22:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:07.45902928 +0000 UTC m=+238.278067681" watchObservedRunningTime="2026-09-10 13:22:09.356672177 +0000 UTC m=+240.175710580"
Sep 10 13:22:11.311082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.310805    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerStarted","Data":"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"}
Sep 10 13:22:11.311082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.310921    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerStarted","Data":"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"}
Sep 10 13:22:11.311082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.310960    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerStarted","Data":"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"}
Sep 10 13:22:11.311082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.310997    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerStarted","Data":"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"}
Sep 10 13:22:11.311082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.311035    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerStarted","Data":"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"}
Sep 10 13:22:11.311082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.311073    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerStarted","Data":"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"}
Sep 10 13:22:11.440230 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.440120    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.335430475 podStartE2EDuration="9.440101808s" totalImagesPullingTime="5.082054824s" totalInitContainerRuntime="22.616509ms" isStatefulPod=false podCreationTimestamp="2026-09-10 13:22:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:11.436583801 +0000 UTC m=+242.255622192" watchObservedRunningTime="2026-09-10 13:22:11.440101808 +0000 UTC m=+242.259140207"
Sep 10 13:22:11.552057 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.552022    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78f4874bf9-jpnpd"]
Sep 10 13:22:11.556833 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.556812    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.569440 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.569422    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 13:22:11.569555 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.569541    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 13:22:11.569657 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.569640    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 13:22:11.569712 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.569639    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 13:22:11.569763 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.569725    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 13:22:11.569969 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.569954    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-l8nfc\""
Sep 10 13:22:11.599072 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.599051    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78f4874bf9-jpnpd"]
Sep 10 13:22:11.669735 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.669700    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-service-ca\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.669735 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.669733    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-console-config\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.669938 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.669823    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhd76\" (UniqueName: \"kubernetes.io/projected/a9b8636c-529b-4527-8d7b-5180db2a9308-kube-api-access-bhd76\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.669938 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.669865    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9b8636c-529b-4527-8d7b-5180db2a9308-console-oauth-config\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.669938 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.669897    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9b8636c-529b-4527-8d7b-5180db2a9308-console-serving-cert\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.670095 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.669975    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-oauth-serving-cert\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.771250 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.771219    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-service-ca\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.771413 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.771268    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-console-config\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.771413 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.771333    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhd76\" (UniqueName: \"kubernetes.io/projected/a9b8636c-529b-4527-8d7b-5180db2a9308-kube-api-access-bhd76\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.771413 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.771381    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9b8636c-529b-4527-8d7b-5180db2a9308-console-oauth-config\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.771601 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.771568    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9b8636c-529b-4527-8d7b-5180db2a9308-console-serving-cert\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.771664 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.771635    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-oauth-serving-cert\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.772059 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.772034    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-service-ca\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.772529 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.772506    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-console-config\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.772619 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.772536    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-oauth-serving-cert\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.774034 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.774012    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9b8636c-529b-4527-8d7b-5180db2a9308-console-oauth-config\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.774126 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.774017    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9b8636c-529b-4527-8d7b-5180db2a9308-console-serving-cert\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.784240 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.784221    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhd76\" (UniqueName: \"kubernetes.io/projected/a9b8636c-529b-4527-8d7b-5180db2a9308-kube-api-access-bhd76\") pod \"console-78f4874bf9-jpnpd\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") " pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:11.867760 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:11.867736    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:12.147002 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:12.146929    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78f4874bf9-jpnpd"]
Sep 10 13:22:12.150375 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:22:12.150338    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b8636c_529b_4527_8d7b_5180db2a9308.slice/crio-7a84ab99651b773c02a298eb10de8734b9012b6ff4243c633071ca31232c96d1 WatchSource:0}: Error finding container 7a84ab99651b773c02a298eb10de8734b9012b6ff4243c633071ca31232c96d1: Status 404 returned error can't find the container with id 7a84ab99651b773c02a298eb10de8734b9012b6ff4243c633071ca31232c96d1
Sep 10 13:22:12.316350 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:12.316309    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78f4874bf9-jpnpd" event={"ID":"a9b8636c-529b-4527-8d7b-5180db2a9308","Type":"ContainerStarted","Data":"7a84ab99651b773c02a298eb10de8734b9012b6ff4243c633071ca31232c96d1"}
Sep 10 13:22:13.213066 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:13.213027    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:22:15.284117 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:15.284007    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9whh7"
Sep 10 13:22:16.336108 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:16.336075    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78f4874bf9-jpnpd" event={"ID":"a9b8636c-529b-4527-8d7b-5180db2a9308","Type":"ContainerStarted","Data":"67d26164b4c52b0c5120389eeba8142d3ac7bbe6b33656cd7489c8bed287c421"}
Sep 10 13:22:21.157350 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:21.157312    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:21.157870 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:21.157387    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:21.868115 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:21.868074    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:21.868306 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:21.868153    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:21.873551 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:21.873527    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:21.980885 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:21.980833    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78f4874bf9-jpnpd" podStartSLOduration=7.683418221 podStartE2EDuration="10.980815717s" totalImagesPullingTime="3.297397496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:22:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:16.39468804 +0000 UTC m=+247.213726440" watchObservedRunningTime="2026-09-10 13:22:21.980815717 +0000 UTC m=+252.799854118"
Sep 10 13:22:21.981043 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:21.980990    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66944f8875-fc8l9"]
Sep 10 13:22:21.984385 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:21.984363    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:21.996855 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:22:21.996829    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"trusted-ca-bundle\" is forbidden: User \"system:node:ip-10-0-135-117.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-135-117.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap"
Sep 10 13:22:21.997521 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:22:21.997497    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"console-66944f8875-fc8l9\" is forbidden: User \"system:node:ip-10-0-135-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-135-117.ec2.internal' and this object" podUID="aa5dde79-310f-4442-a2ac-f7693c46b3ef" pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.026786 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.026757    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66944f8875-fc8l9"]
Sep 10 13:22:22.064486 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.064459    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq9rs\" (UniqueName: \"kubernetes.io/projected/aa5dde79-310f-4442-a2ac-f7693c46b3ef-kube-api-access-lq9rs\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.064640 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.064508    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-oauth-serving-cert\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.064640 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.064613    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-service-ca\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.064752 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.064653    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-config\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.064752 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.064706    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-trusted-ca-bundle\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.064840 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.064785    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-serving-cert\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.064840 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.064817    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-oauth-config\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.166261 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.166172    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq9rs\" (UniqueName: \"kubernetes.io/projected/aa5dde79-310f-4442-a2ac-f7693c46b3ef-kube-api-access-lq9rs\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.166261 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.166227    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-oauth-serving-cert\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.166788 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.166267    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-service-ca\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.166788 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.166287    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-config\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.166788 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.166313    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-trusted-ca-bundle\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.166788 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.166485    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-serving-cert\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.166788 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.166538    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-oauth-config\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.167119 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.167083    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-oauth-serving-cert\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.167119 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.167108    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-service-ca\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.167231 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.167145    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-config\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.169548 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.169522    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-oauth-config\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.169678 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.169567    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-serving-cert\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.180654 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.180635    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq9rs\" (UniqueName: \"kubernetes.io/projected/aa5dde79-310f-4442-a2ac-f7693c46b3ef-kube-api-access-lq9rs\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:22.369598 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:22.369557    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:23.136771 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:23.136720    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 13:22:23.147612 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:23.147563    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-trusted-ca-bundle\") pod \"console-66944f8875-fc8l9\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") " pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:23.195646 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:23.195609    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:24.111305 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:24.111234    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66944f8875-fc8l9"]
Sep 10 13:22:24.133086 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:22:24.133051    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa5dde79_310f_4442_a2ac_f7693c46b3ef.slice/crio-c5e1b9ce148e1aee97b549ae3f1e0a1b9235ff41de98d6ab8cecd36083f1f317 WatchSource:0}: Error finding container c5e1b9ce148e1aee97b549ae3f1e0a1b9235ff41de98d6ab8cecd36083f1f317: Status 404 returned error can't find the container with id c5e1b9ce148e1aee97b549ae3f1e0a1b9235ff41de98d6ab8cecd36083f1f317
Sep 10 13:22:24.374792 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:24.374707    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-ks28m" event={"ID":"75a92c91-c7a6-4b56-b4b1-7437ce6ccb85","Type":"ContainerStarted","Data":"797d73b00817648d7c5d0e6eb2e646223112f9f9900ab09161429e3fa1142f0b"}
Sep 10 13:22:24.375220 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:24.374893    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-ks28m"
Sep 10 13:22:24.376180 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:24.376155    2551 patch_prober.go:28] interesting pod/downloads-85d98c998f-ks28m container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.13:8080/\": dial tcp 10.133.0.13:8080: connect: connection refused" start-of-body=
Sep 10 13:22:24.376301 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:24.376221    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-85d98c998f-ks28m" podUID="75a92c91-c7a6-4b56-b4b1-7437ce6ccb85" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.13:8080/\": dial tcp 10.133.0.13:8080: connect: connection refused"
Sep 10 13:22:24.376818 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:24.376792    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66944f8875-fc8l9" event={"ID":"aa5dde79-310f-4442-a2ac-f7693c46b3ef","Type":"ContainerStarted","Data":"1ccefcec6533ab4fae508e219a0deeab0267968c40073b522d3bc42490e2c40f"}
Sep 10 13:22:24.376898 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:24.376839    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66944f8875-fc8l9" event={"ID":"aa5dde79-310f-4442-a2ac-f7693c46b3ef","Type":"ContainerStarted","Data":"c5e1b9ce148e1aee97b549ae3f1e0a1b9235ff41de98d6ab8cecd36083f1f317"}
Sep 10 13:22:24.436782 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:24.436723    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-ks28m" podStartSLOduration=1.24396132 podStartE2EDuration="18.436703715s" totalImagesPullingTime="17.192742395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:22:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:24.436251829 +0000 UTC m=+255.255290230" watchObservedRunningTime="2026-09-10 13:22:24.436703715 +0000 UTC m=+255.255742114"
Sep 10 13:22:24.484020 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:24.483959    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66944f8875-fc8l9" podStartSLOduration=3.483938248 podStartE2EDuration="3.483938248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:22:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:24.483606166 +0000 UTC m=+255.302644565" watchObservedRunningTime="2026-09-10 13:22:24.483938248 +0000 UTC m=+255.302976710"
Sep 10 13:22:25.396860 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:25.396832    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-ks28m"
Sep 10 13:22:33.196201 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:33.196092    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:33.196201 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:33.196156    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:33.200745 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:33.200724    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:33.418484 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:33.418458    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:22:33.519109 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:33.519079    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78f4874bf9-jpnpd"]
Sep 10 13:22:35.092752 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.092716    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:22:35.107022 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.107000    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 13:22:35.115286 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.115265    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8bc0fa69-2ca1-4cd6-b004-88863086d37d-metrics-certs\") pod \"network-metrics-daemon-xxmwh\" (UID: \"8bc0fa69-2ca1-4cd6-b004-88863086d37d\") " pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:22:35.194004 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.193977    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrm9s\" (UniqueName: \"kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s\") pod \"network-check-target-82cgh\" (UID: \"4c893996-bc6b-46f0-ab3f-b98dcb115908\") " pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:22:35.197720 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.197703    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 13:22:35.212031 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.212008    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 13:22:35.217220 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.217195    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrm9s\" (UniqueName: \"kubernetes.io/projected/4c893996-bc6b-46f0-ab3f-b98dcb115908-kube-api-access-xrm9s\") pod \"network-check-target-82cgh\" (UID: \"4c893996-bc6b-46f0-ab3f-b98dcb115908\") " pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:22:35.294905 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.294882    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:22:35.297728 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.297713    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 13:22:35.307350 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.307328    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68de88a2-c141-4c07-9bfd-9ad2aa25b835-original-pull-secret\") pod \"global-pull-secret-syncer-fnfwb\" (UID: \"68de88a2-c141-4c07-9bfd-9ad2aa25b835\") " pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:22:35.318378 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.318361    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-67hjc\""
Sep 10 13:22:35.325112 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.325097    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4sjdq\""
Sep 10 13:22:35.326152 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.326139    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:22:35.330520 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.330505    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xxmwh"
Sep 10 13:22:35.330591 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.330556    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9hn4m\""
Sep 10 13:22:35.335677 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.335665    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fnfwb"
Sep 10 13:22:35.651163 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:22:35.651131    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c893996_bc6b_46f0_ab3f_b98dcb115908.slice/crio-b529c4e98a38f0a47b68efa34cfd06b9a32b9736f48168db1d90fe8815537fa7 WatchSource:0}: Error finding container b529c4e98a38f0a47b68efa34cfd06b9a32b9736f48168db1d90fe8815537fa7: Status 404 returned error can't find the container with id b529c4e98a38f0a47b68efa34cfd06b9a32b9736f48168db1d90fe8815537fa7
Sep 10 13:22:35.653003 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.652899    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-82cgh"]
Sep 10 13:22:35.700314 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.700289    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xxmwh"]
Sep 10 13:22:35.709325 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:22:35.709302    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bc0fa69_2ca1_4cd6_b004_88863086d37d.slice/crio-ef689ad6c8ac34f5a95b8fa6c51b20fba5339cd0914292d8e4aa1250487bec00 WatchSource:0}: Error finding container ef689ad6c8ac34f5a95b8fa6c51b20fba5339cd0914292d8e4aa1250487bec00: Status 404 returned error can't find the container with id ef689ad6c8ac34f5a95b8fa6c51b20fba5339cd0914292d8e4aa1250487bec00
Sep 10 13:22:35.715510 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:35.715489    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fnfwb"]
Sep 10 13:22:35.718596 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:22:35.718555    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68de88a2_c141_4c07_9bfd_9ad2aa25b835.slice/crio-7a22eac2aee01df9c6059c6688c4f0269055fce38ec8b599b98dd0c47fa862ba WatchSource:0}: Error finding container 7a22eac2aee01df9c6059c6688c4f0269055fce38ec8b599b98dd0c47fa862ba: Status 404 returned error can't find the container with id 7a22eac2aee01df9c6059c6688c4f0269055fce38ec8b599b98dd0c47fa862ba
Sep 10 13:22:36.428740 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:36.428700    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fnfwb" event={"ID":"68de88a2-c141-4c07-9bfd-9ad2aa25b835","Type":"ContainerStarted","Data":"7a22eac2aee01df9c6059c6688c4f0269055fce38ec8b599b98dd0c47fa862ba"}
Sep 10 13:22:36.430188 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:36.430161    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xxmwh" event={"ID":"8bc0fa69-2ca1-4cd6-b004-88863086d37d","Type":"ContainerStarted","Data":"ef689ad6c8ac34f5a95b8fa6c51b20fba5339cd0914292d8e4aa1250487bec00"}
Sep 10 13:22:36.431586 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:36.431538    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-82cgh" event={"ID":"4c893996-bc6b-46f0-ab3f-b98dcb115908","Type":"ContainerStarted","Data":"b529c4e98a38f0a47b68efa34cfd06b9a32b9736f48168db1d90fe8815537fa7"}
Sep 10 13:22:38.442612 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:38.442560    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xxmwh" event={"ID":"8bc0fa69-2ca1-4cd6-b004-88863086d37d","Type":"ContainerStarted","Data":"6c6aab9f17aa66afe520d4afc620de312c91c6af4e3d44d7d39a25d973890b4a"}
Sep 10 13:22:38.443070 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:38.442640    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xxmwh" event={"ID":"8bc0fa69-2ca1-4cd6-b004-88863086d37d","Type":"ContainerStarted","Data":"63267ca333ff390ba4158b3798dcad66523564eb2b4fdfc8f7bddfd09a0a4f72"}
Sep 10 13:22:41.163629 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:41.163599    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:41.168764 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:41.168739    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-8479b9f7c4-68qk5"
Sep 10 13:22:41.223374 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:41.223316    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xxmwh" podStartSLOduration=132.46651546 podStartE2EDuration="2m14.223299221s" totalImagesPullingTime="1.756783761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:38.476484473 +0000 UTC m=+269.295522874" watchObservedRunningTime="2026-09-10 13:22:41.223299221 +0000 UTC m=+272.042337631"
Sep 10 13:22:41.456173 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:41.456089    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fnfwb" event={"ID":"68de88a2-c141-4c07-9bfd-9ad2aa25b835","Type":"ContainerStarted","Data":"3daa8bcf2ff19ead151225bf4d236aab8f8f11decae3a52f990f4e359330925e"}
Sep 10 13:22:41.457621 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:41.457599    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-82cgh" event={"ID":"4c893996-bc6b-46f0-ab3f-b98dcb115908","Type":"ContainerStarted","Data":"b2cdb98a2a6cd6a26053d1967851917417cddc0fe33ba60cca2164b988b922cf"}
Sep 10 13:22:41.457730 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:41.457724    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:22:41.488488 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:41.488445    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fnfwb" podStartSLOduration=129.196068823 podStartE2EDuration="2m14.488431451s" totalImagesPullingTime="5.292362628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:41.488294651 +0000 UTC m=+272.307333051" watchObservedRunningTime="2026-09-10 13:22:41.488431451 +0000 UTC m=+272.307469859"
Sep 10 13:22:41.522894 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:41.522853    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-82cgh" podStartSLOduration=129.173409301 podStartE2EDuration="2m14.522840025s" totalImagesPullingTime="5.349430724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:41.522092958 +0000 UTC m=+272.341131358" watchObservedRunningTime="2026-09-10 13:22:41.522840025 +0000 UTC m=+272.341878425"
Sep 10 13:22:58.539060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.539020    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78f4874bf9-jpnpd" podUID="a9b8636c-529b-4527-8d7b-5180db2a9308" containerName="console" containerID="cri-o://67d26164b4c52b0c5120389eeba8142d3ac7bbe6b33656cd7489c8bed287c421" gracePeriod=15
Sep 10 13:22:58.821666 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.821637    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:58.874624 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.874596    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-oauth-serving-cert\") pod \"a9b8636c-529b-4527-8d7b-5180db2a9308\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") "
Sep 10 13:22:58.874753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.874635    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9b8636c-529b-4527-8d7b-5180db2a9308-console-oauth-config\") pod \"a9b8636c-529b-4527-8d7b-5180db2a9308\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") "
Sep 10 13:22:58.874753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.874689    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-service-ca\") pod \"a9b8636c-529b-4527-8d7b-5180db2a9308\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") "
Sep 10 13:22:58.874753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.874728    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-console-config\") pod \"a9b8636c-529b-4527-8d7b-5180db2a9308\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") "
Sep 10 13:22:58.874899 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.874806    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9b8636c-529b-4527-8d7b-5180db2a9308-console-serving-cert\") pod \"a9b8636c-529b-4527-8d7b-5180db2a9308\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") "
Sep 10 13:22:58.874899 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.874844    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhd76\" (UniqueName: \"kubernetes.io/projected/a9b8636c-529b-4527-8d7b-5180db2a9308-kube-api-access-bhd76\") pod \"a9b8636c-529b-4527-8d7b-5180db2a9308\" (UID: \"a9b8636c-529b-4527-8d7b-5180db2a9308\") "
Sep 10 13:22:58.875109 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.875079    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a9b8636c-529b-4527-8d7b-5180db2a9308" (UID: "a9b8636c-529b-4527-8d7b-5180db2a9308"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:22:58.875231 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.875175    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-service-ca" (OuterVolumeSpecName: "service-ca") pod "a9b8636c-529b-4527-8d7b-5180db2a9308" (UID: "a9b8636c-529b-4527-8d7b-5180db2a9308"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:22:58.875231 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.875194    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-console-config" (OuterVolumeSpecName: "console-config") pod "a9b8636c-529b-4527-8d7b-5180db2a9308" (UID: "a9b8636c-529b-4527-8d7b-5180db2a9308"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:22:58.876935 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.876906    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9b8636c-529b-4527-8d7b-5180db2a9308-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a9b8636c-529b-4527-8d7b-5180db2a9308" (UID: "a9b8636c-529b-4527-8d7b-5180db2a9308"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:22:58.877058 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.877032    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9b8636c-529b-4527-8d7b-5180db2a9308-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a9b8636c-529b-4527-8d7b-5180db2a9308" (UID: "a9b8636c-529b-4527-8d7b-5180db2a9308"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:22:58.877117 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.877053    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9b8636c-529b-4527-8d7b-5180db2a9308-kube-api-access-bhd76" (OuterVolumeSpecName: "kube-api-access-bhd76") pod "a9b8636c-529b-4527-8d7b-5180db2a9308" (UID: "a9b8636c-529b-4527-8d7b-5180db2a9308"). InnerVolumeSpecName "kube-api-access-bhd76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:22:58.976069 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.976045    2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-console-config\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:22:58.976069 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.976065    2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9b8636c-529b-4527-8d7b-5180db2a9308-console-serving-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:22:58.976208 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.976074    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhd76\" (UniqueName: \"kubernetes.io/projected/a9b8636c-529b-4527-8d7b-5180db2a9308-kube-api-access-bhd76\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:22:58.976208 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.976083    2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-oauth-serving-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:22:58.976208 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.976092    2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9b8636c-529b-4527-8d7b-5180db2a9308-console-oauth-config\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:22:58.976208 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:58.976100    2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b8636c-529b-4527-8d7b-5180db2a9308-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:22:59.537940 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:59.537909    2551 generic.go:356] "Generic (PLEG): container finished" podID="a9b8636c-529b-4527-8d7b-5180db2a9308" containerID="67d26164b4c52b0c5120389eeba8142d3ac7bbe6b33656cd7489c8bed287c421" exitCode=0
Sep 10 13:22:59.538097 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:59.537957    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78f4874bf9-jpnpd"
Sep 10 13:22:59.538097 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:59.537990    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78f4874bf9-jpnpd" event={"ID":"a9b8636c-529b-4527-8d7b-5180db2a9308","Type":"ContainerDied","Data":"67d26164b4c52b0c5120389eeba8142d3ac7bbe6b33656cd7489c8bed287c421"}
Sep 10 13:22:59.538097 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:59.538036    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78f4874bf9-jpnpd" event={"ID":"a9b8636c-529b-4527-8d7b-5180db2a9308","Type":"ContainerDied","Data":"7a84ab99651b773c02a298eb10de8734b9012b6ff4243c633071ca31232c96d1"}
Sep 10 13:22:59.538097 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:59.538056    2551 scope.go:117] "RemoveContainer" containerID="67d26164b4c52b0c5120389eeba8142d3ac7bbe6b33656cd7489c8bed287c421"
Sep 10 13:22:59.546690 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:59.546508    2551 scope.go:117] "RemoveContainer" containerID="67d26164b4c52b0c5120389eeba8142d3ac7bbe6b33656cd7489c8bed287c421"
Sep 10 13:22:59.546904 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:22:59.546776    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67d26164b4c52b0c5120389eeba8142d3ac7bbe6b33656cd7489c8bed287c421\": container with ID starting with 67d26164b4c52b0c5120389eeba8142d3ac7bbe6b33656cd7489c8bed287c421 not found: ID does not exist" containerID="67d26164b4c52b0c5120389eeba8142d3ac7bbe6b33656cd7489c8bed287c421"
Sep 10 13:22:59.546904 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:59.546798    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67d26164b4c52b0c5120389eeba8142d3ac7bbe6b33656cd7489c8bed287c421"} err="failed to get container status \"67d26164b4c52b0c5120389eeba8142d3ac7bbe6b33656cd7489c8bed287c421\": rpc error: code = NotFound desc = could not find container \"67d26164b4c52b0c5120389eeba8142d3ac7bbe6b33656cd7489c8bed287c421\": container with ID starting with 67d26164b4c52b0c5120389eeba8142d3ac7bbe6b33656cd7489c8bed287c421 not found: ID does not exist"
Sep 10 13:22:59.565745 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:59.565723    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78f4874bf9-jpnpd"]
Sep 10 13:22:59.571513 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:59.571495    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78f4874bf9-jpnpd"]
Sep 10 13:22:59.709345 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:22:59.709312    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9b8636c-529b-4527-8d7b-5180db2a9308" path="/var/lib/kubelet/pods/a9b8636c-529b-4527-8d7b-5180db2a9308/volumes"
Sep 10 13:23:03.212937 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:03.212899    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:03.295912 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:03.295875    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:03.568608 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:03.568564    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:09.588584 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:09.588543    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:23:09.591804 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:09.591780    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:23:09.595847 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:09.595827    2551 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 13:23:12.462974 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:12.462943    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-82cgh"
Sep 10 13:23:20.885688 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:20.885658    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"]
Sep 10 13:23:20.886047 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:20.885908    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9b8636c-529b-4527-8d7b-5180db2a9308" containerName="console"
Sep 10 13:23:20.886047 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:20.885919    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9b8636c-529b-4527-8d7b-5180db2a9308" containerName="console"
Sep 10 13:23:20.886047 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:20.885973    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9b8636c-529b-4527-8d7b-5180db2a9308" containerName="console"
Sep 10 13:23:20.888172 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:20.888155    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:20.894664 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:20.894646    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 10 13:23:20.894664 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:20.894656    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 10 13:23:20.894823 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:20.894647    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 10 13:23:20.894823 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:20.894648    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 10 13:23:20.895145 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:20.895128    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-hwfg6\""
Sep 10 13:23:20.895428 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:20.895412    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 10 13:23:20.902853 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:20.902836    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 10 13:23:20.911046 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:20.911028    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"]
Sep 10 13:23:21.039058 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.039032    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eec91424-2065-41b5-bb95-b977d2940fef-serving-certs-ca-bundle\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.039203 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.039066    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eec91424-2065-41b5-bb95-b977d2940fef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.039203 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.039090    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eec91424-2065-41b5-bb95-b977d2940fef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.039203 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.039160    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eec91424-2065-41b5-bb95-b977d2940fef-secret-telemeter-client\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.039312 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.039211    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eec91424-2065-41b5-bb95-b977d2940fef-federate-client-tls\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.039312 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.039229    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eec91424-2065-41b5-bb95-b977d2940fef-telemeter-client-tls\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.039312 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.039264    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eec91424-2065-41b5-bb95-b977d2940fef-metrics-client-ca\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.039312 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.039299    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl2r2\" (UniqueName: \"kubernetes.io/projected/eec91424-2065-41b5-bb95-b977d2940fef-kube-api-access-pl2r2\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.140062 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.139984    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eec91424-2065-41b5-bb95-b977d2940fef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.140062 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.140025    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eec91424-2065-41b5-bb95-b977d2940fef-secret-telemeter-client\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.140062 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.140049    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eec91424-2065-41b5-bb95-b977d2940fef-federate-client-tls\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.140062 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.140066    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eec91424-2065-41b5-bb95-b977d2940fef-telemeter-client-tls\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.140370 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.140189    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eec91424-2065-41b5-bb95-b977d2940fef-metrics-client-ca\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.140370 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.140229    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl2r2\" (UniqueName: \"kubernetes.io/projected/eec91424-2065-41b5-bb95-b977d2940fef-kube-api-access-pl2r2\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.140370 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.140261    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eec91424-2065-41b5-bb95-b977d2940fef-serving-certs-ca-bundle\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.140370 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.140291    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eec91424-2065-41b5-bb95-b977d2940fef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.140921 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.140890    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eec91424-2065-41b5-bb95-b977d2940fef-metrics-client-ca\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.141043 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.140981    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eec91424-2065-41b5-bb95-b977d2940fef-serving-certs-ca-bundle\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.141113 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.141101    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eec91424-2065-41b5-bb95-b977d2940fef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.142568 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.142543    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eec91424-2065-41b5-bb95-b977d2940fef-secret-telemeter-client\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.142767 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.142746    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eec91424-2065-41b5-bb95-b977d2940fef-telemeter-client-tls\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.142895 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.142881    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eec91424-2065-41b5-bb95-b977d2940fef-federate-client-tls\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.142966 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.142945    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eec91424-2065-41b5-bb95-b977d2940fef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.152623 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.152592    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl2r2\" (UniqueName: \"kubernetes.io/projected/eec91424-2065-41b5-bb95-b977d2940fef-kube-api-access-pl2r2\") pod \"telemeter-client-55c5f86bf5-fmgwn\" (UID: \"eec91424-2065-41b5-bb95-b977d2940fef\") " pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.197799 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.197750    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"
Sep 10 13:23:21.199990 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.199970    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:23:21.200301 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.200250    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="thanos-sidecar" containerID="cri-o://3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed" gracePeriod=600
Sep 10 13:23:21.200389 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.200320    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624" gracePeriod=600
Sep 10 13:23:21.200389 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.200247    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="prometheus" containerID="cri-o://ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43" gracePeriod=600
Sep 10 13:23:21.200953 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.200309    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="config-reloader" containerID="cri-o://30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c" gracePeriod=600
Sep 10 13:23:21.200953 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.200263    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="kube-rbac-proxy" containerID="cri-o://28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8" gracePeriod=600
Sep 10 13:23:21.200953 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.200280    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="kube-rbac-proxy-web" containerID="cri-o://485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9" gracePeriod=600
Sep 10 13:23:21.438007 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.437984    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.497139 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:23:21.497104    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeec91424_2065_41b5_bb95_b977d2940fef.slice/crio-5b057b3e2bdeda061b418b76c6e824b34a352d1aa9593cd995f67c2af1c90ffb WatchSource:0}: Error finding container 5b057b3e2bdeda061b418b76c6e824b34a352d1aa9593cd995f67c2af1c90ffb: Status 404 returned error can't find the container with id 5b057b3e2bdeda061b418b76c6e824b34a352d1aa9593cd995f67c2af1c90ffb
Sep 10 13:23:21.497284 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.497146    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn"]
Sep 10 13:23:21.499487 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.499472    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 13:23:21.543078 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543050    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-metrics-client-certs\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543085    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-tls\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543112    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-trusted-ca-bundle\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543140    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543174    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b106a48-315d-4133-adb7-f8debe6f0e7c-config-out\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543204    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-serving-certs-ca-bundle\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543465 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543319    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b106a48-315d-4133-adb7-f8debe6f0e7c-tls-assets\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543465 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543371    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-db\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543465 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543403    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-thanos-prometheus-http-client-file\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543690 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543464    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-kube-rbac-proxy\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543690 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543497    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-web-config\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543690 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543532    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2qv8\" (UniqueName: \"kubernetes.io/projected/8b106a48-315d-4133-adb7-f8debe6f0e7c-kube-api-access-d2qv8\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543690 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543598    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-2\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543690 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543632    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-kubelet-serving-ca-bundle\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543690 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543659    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.543690 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543686    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-metrics-client-ca\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.544041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543712    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-0\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.544041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543746    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-1\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.544041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543777    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-grpc-tls\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.544041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543800    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-config\") pod \"8b106a48-315d-4133-adb7-f8debe6f0e7c\" (UID: \"8b106a48-315d-4133-adb7-f8debe6f0e7c\") "
Sep 10 13:23:21.544041 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.543851    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:23:21.544321 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.544056    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.545250 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.545040    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:23:21.545473 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.545445    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:23:21.545584 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.545484    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:23:21.547352 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.545859    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:23:21.547352 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.546229    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:23:21.547352 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.546601    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:23:21.547352 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.546857    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:23:21.547352 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.547038    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:23:21.547352 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.547175    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:23:21.547819 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.547539    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:23:21.547882 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.547836    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:23:21.548049 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.547982    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:23:21.548503 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.548452    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:23:21.548503 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.548466    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b106a48-315d-4133-adb7-f8debe6f0e7c-kube-api-access-d2qv8" (OuterVolumeSpecName: "kube-api-access-d2qv8") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "kube-api-access-d2qv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:23:21.548649 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.548509    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-config" (OuterVolumeSpecName: "config") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:23:21.548878 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.548853    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:23:21.549285 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.549258    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b106a48-315d-4133-adb7-f8debe6f0e7c-config-out" (OuterVolumeSpecName: "config-out") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:23:21.549419 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.549405    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b106a48-315d-4133-adb7-f8debe6f0e7c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:23:21.557845 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.557822    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-web-config" (OuterVolumeSpecName: "web-config") pod "8b106a48-315d-4133-adb7-f8debe6f0e7c" (UID: "8b106a48-315d-4133-adb7-f8debe6f0e7c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:23:21.620060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620033    2551 generic.go:356] "Generic (PLEG): container finished" podID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerID="143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624" exitCode=0
Sep 10 13:23:21.620060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620057    2551 generic.go:356] "Generic (PLEG): container finished" podID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerID="28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8" exitCode=0
Sep 10 13:23:21.620060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620065    2551 generic.go:356] "Generic (PLEG): container finished" podID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerID="485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9" exitCode=0
Sep 10 13:23:21.620276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620073    2551 generic.go:356] "Generic (PLEG): container finished" podID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerID="3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed" exitCode=0
Sep 10 13:23:21.620276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620081    2551 generic.go:356] "Generic (PLEG): container finished" podID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerID="30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c" exitCode=0
Sep 10 13:23:21.620276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620087    2551 generic.go:356] "Generic (PLEG): container finished" podID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerID="ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43" exitCode=0
Sep 10 13:23:21.620276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620106    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.620276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620112    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerDied","Data":"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"}
Sep 10 13:23:21.620276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620181    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerDied","Data":"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"}
Sep 10 13:23:21.620276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620216    2551 scope.go:117] "RemoveContainer" containerID="143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"
Sep 10 13:23:21.620276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620228    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerDied","Data":"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"}
Sep 10 13:23:21.620276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620257    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerDied","Data":"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"}
Sep 10 13:23:21.620696 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620316    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerDied","Data":"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"}
Sep 10 13:23:21.620696 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620359    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerDied","Data":"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"}
Sep 10 13:23:21.620696 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.620404    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b106a48-315d-4133-adb7-f8debe6f0e7c","Type":"ContainerDied","Data":"90147faa30aedfe995a940f4fe239200356115a9243e65fd2fbe6d24931bbaad"}
Sep 10 13:23:21.621462 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.621443    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn" event={"ID":"eec91424-2065-41b5-bb95-b977d2940fef","Type":"ContainerStarted","Data":"5b057b3e2bdeda061b418b76c6e824b34a352d1aa9593cd995f67c2af1c90ffb"}
Sep 10 13:23:21.628648 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.628630    2551 scope.go:117] "RemoveContainer" containerID="28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"
Sep 10 13:23:21.635130 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.635111    2551 scope.go:117] "RemoveContainer" containerID="485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"
Sep 10 13:23:21.642820 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.642798    2551 scope.go:117] "RemoveContainer" containerID="3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"
Sep 10 13:23:21.644938 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.644921    2551 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-kube-rbac-proxy\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645004 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.644941    2551 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-web-config\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645004 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.644951    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2qv8\" (UniqueName: \"kubernetes.io/projected/8b106a48-315d-4133-adb7-f8debe6f0e7c-kube-api-access-d2qv8\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645004 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.644960    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645004 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.644970    2551 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645004 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.644979    2551 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645004 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.644987    2551 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-metrics-client-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645004 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.644995    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645004 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.645003    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645268 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.645011    2551 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-grpc-tls\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645268 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.645019    2551 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-config\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645268 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.645026    2551 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-metrics-client-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645268 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.645034    2551 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645268 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.645042    2551 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645268 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.645050    2551 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b106a48-315d-4133-adb7-f8debe6f0e7c-config-out\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645268 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.645058    2551 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b106a48-315d-4133-adb7-f8debe6f0e7c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645268 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.645065    2551 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b106a48-315d-4133-adb7-f8debe6f0e7c-tls-assets\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645268 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.645074    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b106a48-315d-4133-adb7-f8debe6f0e7c-prometheus-k8s-db\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.645268 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.645082    2551 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b106a48-315d-4133-adb7-f8debe6f0e7c-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:23:21.648805 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.648790    2551 scope.go:117] "RemoveContainer" containerID="30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"
Sep 10 13:23:21.654788 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.654773    2551 scope.go:117] "RemoveContainer" containerID="ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"
Sep 10 13:23:21.660941 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.660919    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:23:21.661998 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.661979    2551 scope.go:117] "RemoveContainer" containerID="50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"
Sep 10 13:23:21.667933 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.667918    2551 scope.go:117] "RemoveContainer" containerID="143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"
Sep 10 13:23:21.668180 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:23:21.668160    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624\": container with ID starting with 143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624 not found: ID does not exist" containerID="143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"
Sep 10 13:23:21.668262 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.668192    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"} err="failed to get container status \"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624\": rpc error: code = NotFound desc = could not find container \"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624\": container with ID starting with 143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624 not found: ID does not exist"
Sep 10 13:23:21.668262 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.668215    2551 scope.go:117] "RemoveContainer" containerID="28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"
Sep 10 13:23:21.668470 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:23:21.668453    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8\": container with ID starting with 28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8 not found: ID does not exist" containerID="28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"
Sep 10 13:23:21.668507 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.668476    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"} err="failed to get container status \"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8\": rpc error: code = NotFound desc = could not find container \"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8\": container with ID starting with 28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8 not found: ID does not exist"
Sep 10 13:23:21.668507 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.668490    2551 scope.go:117] "RemoveContainer" containerID="485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"
Sep 10 13:23:21.668702 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:23:21.668687    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9\": container with ID starting with 485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9 not found: ID does not exist" containerID="485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"
Sep 10 13:23:21.668763 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.668707    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"} err="failed to get container status \"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9\": rpc error: code = NotFound desc = could not find container \"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9\": container with ID starting with 485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9 not found: ID does not exist"
Sep 10 13:23:21.668763 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.668721    2551 scope.go:117] "RemoveContainer" containerID="3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"
Sep 10 13:23:21.668913 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:23:21.668898    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed\": container with ID starting with 3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed not found: ID does not exist" containerID="3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"
Sep 10 13:23:21.668954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.668916    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"} err="failed to get container status \"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed\": rpc error: code = NotFound desc = could not find container \"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed\": container with ID starting with 3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed not found: ID does not exist"
Sep 10 13:23:21.668954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.668929    2551 scope.go:117] "RemoveContainer" containerID="30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"
Sep 10 13:23:21.669140 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:23:21.669122    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c\": container with ID starting with 30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c not found: ID does not exist" containerID="30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"
Sep 10 13:23:21.669183 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.669143    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"} err="failed to get container status \"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c\": rpc error: code = NotFound desc = could not find container \"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c\": container with ID starting with 30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c not found: ID does not exist"
Sep 10 13:23:21.669183 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.669154    2551 scope.go:117] "RemoveContainer" containerID="ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"
Sep 10 13:23:21.669389 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:23:21.669372    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43\": container with ID starting with ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43 not found: ID does not exist" containerID="ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"
Sep 10 13:23:21.669446 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.669390    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"} err="failed to get container status \"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43\": rpc error: code = NotFound desc = could not find container \"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43\": container with ID starting with ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43 not found: ID does not exist"
Sep 10 13:23:21.669446 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.669401    2551 scope.go:117] "RemoveContainer" containerID="50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"
Sep 10 13:23:21.669650 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:23:21.669634    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e\": container with ID starting with 50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e not found: ID does not exist" containerID="50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"
Sep 10 13:23:21.669650 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.669650    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"} err="failed to get container status \"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e\": rpc error: code = NotFound desc = could not find container \"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e\": container with ID starting with 50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e not found: ID does not exist"
Sep 10 13:23:21.669650 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.669662    2551 scope.go:117] "RemoveContainer" containerID="143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"
Sep 10 13:23:21.669899 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.669881    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"} err="failed to get container status \"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624\": rpc error: code = NotFound desc = could not find container \"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624\": container with ID starting with 143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624 not found: ID does not exist"
Sep 10 13:23:21.669945 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.669900    2551 scope.go:117] "RemoveContainer" containerID="28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"
Sep 10 13:23:21.670121 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.670106    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"} err="failed to get container status \"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8\": rpc error: code = NotFound desc = could not find container \"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8\": container with ID starting with 28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8 not found: ID does not exist"
Sep 10 13:23:21.670170 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.670122    2551 scope.go:117] "RemoveContainer" containerID="485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"
Sep 10 13:23:21.670332 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.670317    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"} err="failed to get container status \"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9\": rpc error: code = NotFound desc = could not find container \"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9\": container with ID starting with 485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9 not found: ID does not exist"
Sep 10 13:23:21.670371 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.670332    2551 scope.go:117] "RemoveContainer" containerID="3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"
Sep 10 13:23:21.670560 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.670535    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"} err="failed to get container status \"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed\": rpc error: code = NotFound desc = could not find container \"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed\": container with ID starting with 3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed not found: ID does not exist"
Sep 10 13:23:21.670707 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.670561    2551 scope.go:117] "RemoveContainer" containerID="30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"
Sep 10 13:23:21.670934 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.670905    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"} err="failed to get container status \"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c\": rpc error: code = NotFound desc = could not find container \"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c\": container with ID starting with 30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c not found: ID does not exist"
Sep 10 13:23:21.671017 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.670936    2551 scope.go:117] "RemoveContainer" containerID="ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"
Sep 10 13:23:21.671292 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.671260    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"} err="failed to get container status \"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43\": rpc error: code = NotFound desc = could not find container \"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43\": container with ID starting with ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43 not found: ID does not exist"
Sep 10 13:23:21.671292 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.671285    2551 scope.go:117] "RemoveContainer" containerID="50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"
Sep 10 13:23:21.671606 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.671569    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"} err="failed to get container status \"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e\": rpc error: code = NotFound desc = could not find container \"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e\": container with ID starting with 50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e not found: ID does not exist"
Sep 10 13:23:21.671670 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.671607    2551 scope.go:117] "RemoveContainer" containerID="143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"
Sep 10 13:23:21.671860 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.671842    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"} err="failed to get container status \"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624\": rpc error: code = NotFound desc = could not find container \"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624\": container with ID starting with 143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624 not found: ID does not exist"
Sep 10 13:23:21.671907 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.671861    2551 scope.go:117] "RemoveContainer" containerID="28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"
Sep 10 13:23:21.672119 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.672102    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"} err="failed to get container status \"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8\": rpc error: code = NotFound desc = could not find container \"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8\": container with ID starting with 28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8 not found: ID does not exist"
Sep 10 13:23:21.672182 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.672121    2551 scope.go:117] "RemoveContainer" containerID="485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"
Sep 10 13:23:21.672380 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.672360    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"} err="failed to get container status \"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9\": rpc error: code = NotFound desc = could not find container \"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9\": container with ID starting with 485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9 not found: ID does not exist"
Sep 10 13:23:21.672429 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.672381    2551 scope.go:117] "RemoveContainer" containerID="3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"
Sep 10 13:23:21.672602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.672584    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"} err="failed to get container status \"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed\": rpc error: code = NotFound desc = could not find container \"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed\": container with ID starting with 3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed not found: ID does not exist"
Sep 10 13:23:21.672659 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.672602    2551 scope.go:117] "RemoveContainer" containerID="30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"
Sep 10 13:23:21.672716 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.672697    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:23:21.672818 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.672802    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"} err="failed to get container status \"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c\": rpc error: code = NotFound desc = could not find container \"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c\": container with ID starting with 30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c not found: ID does not exist"
Sep 10 13:23:21.672871 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.672818    2551 scope.go:117] "RemoveContainer" containerID="ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"
Sep 10 13:23:21.673034 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.673014    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"} err="failed to get container status \"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43\": rpc error: code = NotFound desc = could not find container \"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43\": container with ID starting with ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43 not found: ID does not exist"
Sep 10 13:23:21.673109 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.673036    2551 scope.go:117] "RemoveContainer" containerID="50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"
Sep 10 13:23:21.673404 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.673381    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"} err="failed to get container status \"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e\": rpc error: code = NotFound desc = could not find container \"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e\": container with ID starting with 50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e not found: ID does not exist"
Sep 10 13:23:21.673404 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.673404    2551 scope.go:117] "RemoveContainer" containerID="143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"
Sep 10 13:23:21.673689 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.673672    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"} err="failed to get container status \"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624\": rpc error: code = NotFound desc = could not find container \"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624\": container with ID starting with 143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624 not found: ID does not exist"
Sep 10 13:23:21.673689 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.673687    2551 scope.go:117] "RemoveContainer" containerID="28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"
Sep 10 13:23:21.673926 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.673909    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"} err="failed to get container status \"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8\": rpc error: code = NotFound desc = could not find container \"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8\": container with ID starting with 28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8 not found: ID does not exist"
Sep 10 13:23:21.673975 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.673927    2551 scope.go:117] "RemoveContainer" containerID="485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"
Sep 10 13:23:21.674134 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.674117    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"} err="failed to get container status \"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9\": rpc error: code = NotFound desc = could not find container \"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9\": container with ID starting with 485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9 not found: ID does not exist"
Sep 10 13:23:21.674134 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.674131    2551 scope.go:117] "RemoveContainer" containerID="3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"
Sep 10 13:23:21.674355 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.674341    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"} err="failed to get container status \"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed\": rpc error: code = NotFound desc = could not find container \"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed\": container with ID starting with 3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed not found: ID does not exist"
Sep 10 13:23:21.674355 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.674354    2551 scope.go:117] "RemoveContainer" containerID="30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"
Sep 10 13:23:21.674529 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.674516    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"} err="failed to get container status \"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c\": rpc error: code = NotFound desc = could not find container \"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c\": container with ID starting with 30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c not found: ID does not exist"
Sep 10 13:23:21.674586 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.674528    2551 scope.go:117] "RemoveContainer" containerID="ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"
Sep 10 13:23:21.674740 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.674725    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"} err="failed to get container status \"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43\": rpc error: code = NotFound desc = could not find container \"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43\": container with ID starting with ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43 not found: ID does not exist"
Sep 10 13:23:21.674740 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.674739    2551 scope.go:117] "RemoveContainer" containerID="50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"
Sep 10 13:23:21.674926 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.674908    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"} err="failed to get container status \"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e\": rpc error: code = NotFound desc = could not find container \"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e\": container with ID starting with 50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e not found: ID does not exist"
Sep 10 13:23:21.674968 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.674927    2551 scope.go:117] "RemoveContainer" containerID="143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"
Sep 10 13:23:21.675145 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.675126    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"} err="failed to get container status \"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624\": rpc error: code = NotFound desc = could not find container \"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624\": container with ID starting with 143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624 not found: ID does not exist"
Sep 10 13:23:21.675197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.675146    2551 scope.go:117] "RemoveContainer" containerID="28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"
Sep 10 13:23:21.675386 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.675369    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"} err="failed to get container status \"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8\": rpc error: code = NotFound desc = could not find container \"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8\": container with ID starting with 28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8 not found: ID does not exist"
Sep 10 13:23:21.675433 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.675386    2551 scope.go:117] "RemoveContainer" containerID="485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"
Sep 10 13:23:21.675621 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.675602    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"} err="failed to get container status \"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9\": rpc error: code = NotFound desc = could not find container \"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9\": container with ID starting with 485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9 not found: ID does not exist"
Sep 10 13:23:21.675669 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.675622    2551 scope.go:117] "RemoveContainer" containerID="3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"
Sep 10 13:23:21.675832 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.675813    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"} err="failed to get container status \"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed\": rpc error: code = NotFound desc = could not find container \"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed\": container with ID starting with 3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed not found: ID does not exist"
Sep 10 13:23:21.675900 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.675834    2551 scope.go:117] "RemoveContainer" containerID="30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"
Sep 10 13:23:21.676050 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.676035    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"} err="failed to get container status \"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c\": rpc error: code = NotFound desc = could not find container \"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c\": container with ID starting with 30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c not found: ID does not exist"
Sep 10 13:23:21.676099 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.676049    2551 scope.go:117] "RemoveContainer" containerID="ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"
Sep 10 13:23:21.676251 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.676233    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"} err="failed to get container status \"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43\": rpc error: code = NotFound desc = could not find container \"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43\": container with ID starting with ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43 not found: ID does not exist"
Sep 10 13:23:21.676290 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.676252    2551 scope.go:117] "RemoveContainer" containerID="50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"
Sep 10 13:23:21.676445 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.676429    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"} err="failed to get container status \"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e\": rpc error: code = NotFound desc = could not find container \"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e\": container with ID starting with 50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e not found: ID does not exist"
Sep 10 13:23:21.676511 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.676447    2551 scope.go:117] "RemoveContainer" containerID="143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"
Sep 10 13:23:21.676669 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.676648    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624"} err="failed to get container status \"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624\": rpc error: code = NotFound desc = could not find container \"143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624\": container with ID starting with 143fdf5faf798ed0eab1df8545f37539aa648e5249d4ecb7227dbd41a8ca2624 not found: ID does not exist"
Sep 10 13:23:21.676721 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.676670    2551 scope.go:117] "RemoveContainer" containerID="28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"
Sep 10 13:23:21.676867 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.676849    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8"} err="failed to get container status \"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8\": rpc error: code = NotFound desc = could not find container \"28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8\": container with ID starting with 28d4c4ff2b55401d68ddf0cefae0427cbb9faa79b0c29468bcfc13eb52f5c8f8 not found: ID does not exist"
Sep 10 13:23:21.676931 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.676868    2551 scope.go:117] "RemoveContainer" containerID="485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"
Sep 10 13:23:21.677067 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.677053    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9"} err="failed to get container status \"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9\": rpc error: code = NotFound desc = could not find container \"485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9\": container with ID starting with 485c5b81a9c715b698d42b80e003e00de87af8269083f7732a9411b7bd7360b9 not found: ID does not exist"
Sep 10 13:23:21.677113 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.677068    2551 scope.go:117] "RemoveContainer" containerID="3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"
Sep 10 13:23:21.677257 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.677242    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed"} err="failed to get container status \"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed\": rpc error: code = NotFound desc = could not find container \"3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed\": container with ID starting with 3ffc8ef723c4f1f27fc57d69727bb43d6b6705920f20e3c36c1e414398f6c3ed not found: ID does not exist"
Sep 10 13:23:21.677301 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.677257    2551 scope.go:117] "RemoveContainer" containerID="30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"
Sep 10 13:23:21.677447 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.677431    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c"} err="failed to get container status \"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c\": rpc error: code = NotFound desc = could not find container \"30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c\": container with ID starting with 30d818af1047944b08ba0d925771cd7e63c9180bcd16bceffb9d85f5f266705c not found: ID does not exist"
Sep 10 13:23:21.677515 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.677449    2551 scope.go:117] "RemoveContainer" containerID="ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"
Sep 10 13:23:21.677694 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.677680    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43"} err="failed to get container status \"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43\": rpc error: code = NotFound desc = could not find container \"ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43\": container with ID starting with ef2a9a5bf7e2567132bb83fd602958737eab33aa8680fbbb0f132f6f4d477c43 not found: ID does not exist"
Sep 10 13:23:21.677745 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.677694    2551 scope.go:117] "RemoveContainer" containerID="50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"
Sep 10 13:23:21.677890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.677872    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e"} err="failed to get container status \"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e\": rpc error: code = NotFound desc = could not find container \"50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e\": container with ID starting with 50c76ca535d247d4b132583a9bbd7f71bc9539f4c38fa6bd56b5723bf8674a9e not found: ID does not exist"
Sep 10 13:23:21.708460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.708397    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" path="/var/lib/kubelet/pods/8b106a48-315d-4133-adb7-f8debe6f0e7c/volumes"
Sep 10 13:23:21.735962 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.735935    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:23:21.736215 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736199    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="init-config-reloader"
Sep 10 13:23:21.736288 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736219    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="init-config-reloader"
Sep 10 13:23:21.736288 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736231    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="kube-rbac-proxy-thanos"
Sep 10 13:23:21.736288 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736240    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="kube-rbac-proxy-thanos"
Sep 10 13:23:21.736288 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736249    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="kube-rbac-proxy-web"
Sep 10 13:23:21.736288 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736258    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="kube-rbac-proxy-web"
Sep 10 13:23:21.736288 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736270    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="thanos-sidecar"
Sep 10 13:23:21.736288 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736279    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="thanos-sidecar"
Sep 10 13:23:21.736663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736299    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="config-reloader"
Sep 10 13:23:21.736663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736307    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="config-reloader"
Sep 10 13:23:21.736663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736324    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="kube-rbac-proxy"
Sep 10 13:23:21.736663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736332    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="kube-rbac-proxy"
Sep 10 13:23:21.736663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736346    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="prometheus"
Sep 10 13:23:21.736663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736354    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="prometheus"
Sep 10 13:23:21.736663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736416    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="prometheus"
Sep 10 13:23:21.736663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736428    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="kube-rbac-proxy-thanos"
Sep 10 13:23:21.736663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736438    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="kube-rbac-proxy-web"
Sep 10 13:23:21.736663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736448    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="kube-rbac-proxy"
Sep 10 13:23:21.736663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736457    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="thanos-sidecar"
Sep 10 13:23:21.736663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.736466    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b106a48-315d-4133-adb7-f8debe6f0e7c" containerName="config-reloader"
Sep 10 13:23:21.740215 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.740198    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.744425 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.744409    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 13:23:21.746004 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.745984    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 13:23:21.747261 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.747243    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 13:23:21.747753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.747496    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 13:23:21.747753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.747502    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 13:23:21.747753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.747566    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 13:23:21.747753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.747569    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 13:23:21.747753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.747621    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 13:23:21.748116 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.748097    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 13:23:21.748227 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.748211    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 13:23:21.748283 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.748234    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ef86qorjuqku3\""
Sep 10 13:23:21.748338 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.748214    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 13:23:21.748338 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.748218    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 13:23:21.748542 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.748526    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rjxd5\""
Sep 10 13:23:21.756814 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.754248    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 13:23:21.762977 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.762960    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 13:23:21.779513 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.779492    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:23:21.845870 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.845841    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.845870 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.845873    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/87d2daa4-8508-4aab-9993-46a044dc0f16-config-out\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846028 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.845894    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846028 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.845918    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846028 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.845983    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846028 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846011    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-config\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846159 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846036    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846159 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846070    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846159 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846086    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846159 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846107    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846159 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846124    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn87w\" (UniqueName: \"kubernetes.io/projected/87d2daa4-8508-4aab-9993-46a044dc0f16-kube-api-access-hn87w\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846159 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846141    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846159 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846154    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846363 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846173    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846363 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846219    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846363 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846238    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846363 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846255    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846363 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846291    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/87d2daa4-8508-4aab-9993-46a044dc0f16-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846363 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846318    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.846363 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.846350    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-web-config\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947168 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947140    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947168 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947170    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947188    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947204    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947220    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947234    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947253    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/87d2daa4-8508-4aab-9993-46a044dc0f16-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947280    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947305    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-web-config\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947335    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947353    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/87d2daa4-8508-4aab-9993-46a044dc0f16-config-out\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947375    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947411    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947436    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947456    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-config\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947477    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947504    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.947660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.947535    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.948463 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.948033    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.948463 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.948213    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.948463 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.948275    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.948463 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.948283    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.948720 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.948461    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.949953 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.949001    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.949953 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.949056    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn87w\" (UniqueName: \"kubernetes.io/projected/87d2daa4-8508-4aab-9993-46a044dc0f16-kube-api-access-hn87w\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.949953 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.949709    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.949953 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.949909    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.951488 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.951463    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/87d2daa4-8508-4aab-9993-46a044dc0f16-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.951656 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.951592    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.951755 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.951708    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-config\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.952389 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.951968    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/87d2daa4-8508-4aab-9993-46a044dc0f16-config-out\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.952389 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.952237    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-web-config\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.952389 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.952238    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/87d2daa4-8508-4aab-9993-46a044dc0f16-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.952389 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.952303    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.952389 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.952357    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.952675 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.952535    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.952779 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.952759    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.952954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.952938    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.953441 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.953424    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/87d2daa4-8508-4aab-9993-46a044dc0f16-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:21.963998 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:21.963947    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn87w\" (UniqueName: \"kubernetes.io/projected/87d2daa4-8508-4aab-9993-46a044dc0f16-kube-api-access-hn87w\") pod \"prometheus-k8s-0\" (UID: \"87d2daa4-8508-4aab-9993-46a044dc0f16\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:22.049111 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:22.049088    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:22.332834 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:22.332805    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:23:22.334451 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:23:22.334418    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87d2daa4_8508_4aab_9993_46a044dc0f16.slice/crio-2849318db18d5e62d2871e128eaed3c516fef0c0ceaa4b4393abfdbd72448e54 WatchSource:0}: Error finding container 2849318db18d5e62d2871e128eaed3c516fef0c0ceaa4b4393abfdbd72448e54: Status 404 returned error can't find the container with id 2849318db18d5e62d2871e128eaed3c516fef0c0ceaa4b4393abfdbd72448e54
Sep 10 13:23:22.628817 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:22.628731    2551 generic.go:356] "Generic (PLEG): container finished" podID="87d2daa4-8508-4aab-9993-46a044dc0f16" containerID="544c1a03e19ce4590bfa2f86648ee35e0576e0b904b2c7f4afa296cb5f7cd6d8" exitCode=0
Sep 10 13:23:22.628972 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:22.628809    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"87d2daa4-8508-4aab-9993-46a044dc0f16","Type":"ContainerDied","Data":"544c1a03e19ce4590bfa2f86648ee35e0576e0b904b2c7f4afa296cb5f7cd6d8"}
Sep 10 13:23:22.628972 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:22.628900    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"87d2daa4-8508-4aab-9993-46a044dc0f16","Type":"ContainerStarted","Data":"2849318db18d5e62d2871e128eaed3c516fef0c0ceaa4b4393abfdbd72448e54"}
Sep 10 13:23:23.635919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:23.635893    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"87d2daa4-8508-4aab-9993-46a044dc0f16","Type":"ContainerStarted","Data":"2489edba6cdfbdaf42614369f2c804b62e980cfeba749f5f5d1ec8fc7b1d11f1"}
Sep 10 13:23:23.636252 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:23.635969    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"87d2daa4-8508-4aab-9993-46a044dc0f16","Type":"ContainerStarted","Data":"7adeeedc6b088f43f1a29b6b296bce2e19c01b764acb2c4ec241575ace21f860"}
Sep 10 13:23:23.636252 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:23.636021    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"87d2daa4-8508-4aab-9993-46a044dc0f16","Type":"ContainerStarted","Data":"d5b1b4e96f6514346a3b4a95e3c42a5fb6256890a89189554e337204c3fa8c87"}
Sep 10 13:23:23.636252 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:23.636064    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"87d2daa4-8508-4aab-9993-46a044dc0f16","Type":"ContainerStarted","Data":"6fb5bebddcdf4b22fc91153e3c8a6692b29279fdc99267c0804fdbcc14a6e6aa"}
Sep 10 13:23:23.636252 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:23.636105    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"87d2daa4-8508-4aab-9993-46a044dc0f16","Type":"ContainerStarted","Data":"db3a0e38de52682da3a96897666611c087a7dfd35ef0ff71c73e8e1d7d19404f"}
Sep 10 13:23:23.636252 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:23.636144    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"87d2daa4-8508-4aab-9993-46a044dc0f16","Type":"ContainerStarted","Data":"59ae507f636f6d17f3e87ad21cb649d5e21b18cabfc6f1b1d77e9f96524acdc3"}
Sep 10 13:23:23.695749 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:23.695706    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.6696222990000003 podStartE2EDuration="2.695690838s" totalImagesPullingTime="0s" totalInitContainerRuntime="26.068539ms" isStatefulPod=false podCreationTimestamp="2026-09-10 13:23:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:23:23.695441464 +0000 UTC m=+314.514479864" watchObservedRunningTime="2026-09-10 13:23:23.695690838 +0000 UTC m=+314.514729238"
Sep 10 13:23:24.641702 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:24.641654    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn" event={"ID":"eec91424-2065-41b5-bb95-b977d2940fef","Type":"ContainerStarted","Data":"34f0ec2fae71c5c00708d9b2c861dbaad8885a2dc8033c1ebff1c87f8dc9cf30"}
Sep 10 13:23:24.642114 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:24.641725    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn" event={"ID":"eec91424-2065-41b5-bb95-b977d2940fef","Type":"ContainerStarted","Data":"d8ccb6f0671cdd7a7d4e536f632c66e333054249a62ee428a144c028ac3e4ec4"}
Sep 10 13:23:24.642114 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:24.641756    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn" event={"ID":"eec91424-2065-41b5-bb95-b977d2940fef","Type":"ContainerStarted","Data":"a1fca43d96ecc7c666241c031e5eb6b203ed559c157dc9764113dfa90e501114"}
Sep 10 13:23:24.671644 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:24.671597    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-55c5f86bf5-fmgwn" podStartSLOduration=2.582435888 podStartE2EDuration="4.671562231s" totalImagesPullingTime="2.089126343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:23:24.669935315 +0000 UTC m=+315.488973715" watchObservedRunningTime="2026-09-10 13:23:24.671562231 +0000 UTC m=+315.490600630"
Sep 10 13:23:24.831662 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:24.831631    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66d45ddcb5-z4xr2"]
Sep 10 13:23:24.834025 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:24.834007    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:24.852406 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:24.852385    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66d45ddcb5-z4xr2"]
Sep 10 13:23:24.977261 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:24.977229    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-oauth-serving-cert\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:24.977409 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:24.977278    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-serving-cert\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:24.977409 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:24.977293    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-oauth-config\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:24.977409 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:24.977383    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jskss\" (UniqueName: \"kubernetes.io/projected/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-kube-api-access-jskss\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:24.977530 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:24.977412    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-service-ca\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:24.977530 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:24.977450    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-config\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:24.977530 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:24.977464    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-trusted-ca-bundle\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.078421 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.078394    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-config\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.078594 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.078429    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-trusted-ca-bundle\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.078594 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.078454    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-oauth-serving-cert\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.078678 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.078615    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-serving-cert\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.078678 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.078654    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-oauth-config\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.078753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.078701    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jskss\" (UniqueName: \"kubernetes.io/projected/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-kube-api-access-jskss\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.078753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.078733    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-service-ca\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.079258 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.079231    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-config\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.079378 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.079261    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-oauth-serving-cert\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.079378 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.079365    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-service-ca\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.079567 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.079544    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-trusted-ca-bundle\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.081216 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.081195    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-serving-cert\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.081300 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.081283    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-oauth-config\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.091936 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.091917    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jskss\" (UniqueName: \"kubernetes.io/projected/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-kube-api-access-jskss\") pod \"console-66d45ddcb5-z4xr2\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") " pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.142917 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.142894    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:25.419389 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.419336    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66d45ddcb5-z4xr2"]
Sep 10 13:23:25.421353 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:23:25.421329    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d84b92_63b2_41ad_b6c1_9edaaa494e3b.slice/crio-b049743dbd52a6981b85358b5b385dbf712727a501200112019edd60d4e26a18 WatchSource:0}: Error finding container b049743dbd52a6981b85358b5b385dbf712727a501200112019edd60d4e26a18: Status 404 returned error can't find the container with id b049743dbd52a6981b85358b5b385dbf712727a501200112019edd60d4e26a18
Sep 10 13:23:25.647790 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.647751    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d45ddcb5-z4xr2" event={"ID":"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b","Type":"ContainerStarted","Data":"5d4132df0a2c381d9714e9e7e950821a688183ad1b67aff7e2751b0ef2e69932"}
Sep 10 13:23:25.648193 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.647805    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d45ddcb5-z4xr2" event={"ID":"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b","Type":"ContainerStarted","Data":"b049743dbd52a6981b85358b5b385dbf712727a501200112019edd60d4e26a18"}
Sep 10 13:23:25.719386 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:25.719289    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66d45ddcb5-z4xr2" podStartSLOduration=1.719275102 podStartE2EDuration="1.719275102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:23:25.718749145 +0000 UTC m=+316.537787546" watchObservedRunningTime="2026-09-10 13:23:25.719275102 +0000 UTC m=+316.538313501"
Sep 10 13:23:27.049480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:27.049447    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:23:35.143319 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:35.143282    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:35.143796 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:35.143343    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:35.147671 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:35.147647    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:35.688862 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:35.688836    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:23:35.789797 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:23:35.789765    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66944f8875-fc8l9"]
Sep 10 13:24:00.808334 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:00.808295    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66944f8875-fc8l9" podUID="aa5dde79-310f-4442-a2ac-f7693c46b3ef" containerName="console" containerID="cri-o://1ccefcec6533ab4fae508e219a0deeab0267968c40073b522d3bc42490e2c40f" gracePeriod=15
Sep 10 13:24:01.047189 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.047167    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:24:01.139872 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.139803    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lq9rs\" (UniqueName: \"kubernetes.io/projected/aa5dde79-310f-4442-a2ac-f7693c46b3ef-kube-api-access-lq9rs\") pod \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") "
Sep 10 13:24:01.139872 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.139843    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-serving-cert\") pod \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") "
Sep 10 13:24:01.139872 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.139868    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-config\") pod \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") "
Sep 10 13:24:01.140126 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.139889    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-service-ca\") pod \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") "
Sep 10 13:24:01.140126 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.139952    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-oauth-serving-cert\") pod \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") "
Sep 10 13:24:01.140126 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.140050    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-trusted-ca-bundle\") pod \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") "
Sep 10 13:24:01.140126 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.140083    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-oauth-config\") pod \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\" (UID: \"aa5dde79-310f-4442-a2ac-f7693c46b3ef\") "
Sep 10 13:24:01.140356 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.140285    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-config" (OuterVolumeSpecName: "console-config") pod "aa5dde79-310f-4442-a2ac-f7693c46b3ef" (UID: "aa5dde79-310f-4442-a2ac-f7693c46b3ef"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:24:01.140356 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.140299    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-service-ca" (OuterVolumeSpecName: "service-ca") pod "aa5dde79-310f-4442-a2ac-f7693c46b3ef" (UID: "aa5dde79-310f-4442-a2ac-f7693c46b3ef"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:24:01.140356 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.140333    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "aa5dde79-310f-4442-a2ac-f7693c46b3ef" (UID: "aa5dde79-310f-4442-a2ac-f7693c46b3ef"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:24:01.140501 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.140419    2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-config\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:24:01.140501 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.140436    2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:24:01.140501 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.140449    2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-oauth-serving-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:24:01.140501 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.140465    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "aa5dde79-310f-4442-a2ac-f7693c46b3ef" (UID: "aa5dde79-310f-4442-a2ac-f7693c46b3ef"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:24:01.142037 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.142012    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "aa5dde79-310f-4442-a2ac-f7693c46b3ef" (UID: "aa5dde79-310f-4442-a2ac-f7693c46b3ef"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:24:01.142279 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.142243    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa5dde79-310f-4442-a2ac-f7693c46b3ef-kube-api-access-lq9rs" (OuterVolumeSpecName: "kube-api-access-lq9rs") pod "aa5dde79-310f-4442-a2ac-f7693c46b3ef" (UID: "aa5dde79-310f-4442-a2ac-f7693c46b3ef"). InnerVolumeSpecName "kube-api-access-lq9rs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:24:01.142401 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.142290    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "aa5dde79-310f-4442-a2ac-f7693c46b3ef" (UID: "aa5dde79-310f-4442-a2ac-f7693c46b3ef"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:24:01.240967 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.240934    2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa5dde79-310f-4442-a2ac-f7693c46b3ef-trusted-ca-bundle\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:24:01.240967 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.240962    2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-oauth-config\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:24:01.241128 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.240976    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq9rs\" (UniqueName: \"kubernetes.io/projected/aa5dde79-310f-4442-a2ac-f7693c46b3ef-kube-api-access-lq9rs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:24:01.241128 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.240988    2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa5dde79-310f-4442-a2ac-f7693c46b3ef-console-serving-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:24:01.777591 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.777544    2551 generic.go:356] "Generic (PLEG): container finished" podID="aa5dde79-310f-4442-a2ac-f7693c46b3ef" containerID="1ccefcec6533ab4fae508e219a0deeab0267968c40073b522d3bc42490e2c40f" exitCode=0
Sep 10 13:24:01.777740 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.777614    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66944f8875-fc8l9"
Sep 10 13:24:01.777740 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.777626    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66944f8875-fc8l9" event={"ID":"aa5dde79-310f-4442-a2ac-f7693c46b3ef","Type":"ContainerDied","Data":"1ccefcec6533ab4fae508e219a0deeab0267968c40073b522d3bc42490e2c40f"}
Sep 10 13:24:01.777740 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.777673    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66944f8875-fc8l9" event={"ID":"aa5dde79-310f-4442-a2ac-f7693c46b3ef","Type":"ContainerDied","Data":"c5e1b9ce148e1aee97b549ae3f1e0a1b9235ff41de98d6ab8cecd36083f1f317"}
Sep 10 13:24:01.777740 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.777693    2551 scope.go:117] "RemoveContainer" containerID="1ccefcec6533ab4fae508e219a0deeab0267968c40073b522d3bc42490e2c40f"
Sep 10 13:24:01.785594 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.785562    2551 scope.go:117] "RemoveContainer" containerID="1ccefcec6533ab4fae508e219a0deeab0267968c40073b522d3bc42490e2c40f"
Sep 10 13:24:01.785836 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:24:01.785819    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ccefcec6533ab4fae508e219a0deeab0267968c40073b522d3bc42490e2c40f\": container with ID starting with 1ccefcec6533ab4fae508e219a0deeab0267968c40073b522d3bc42490e2c40f not found: ID does not exist" containerID="1ccefcec6533ab4fae508e219a0deeab0267968c40073b522d3bc42490e2c40f"
Sep 10 13:24:01.785903 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.785846    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ccefcec6533ab4fae508e219a0deeab0267968c40073b522d3bc42490e2c40f"} err="failed to get container status \"1ccefcec6533ab4fae508e219a0deeab0267968c40073b522d3bc42490e2c40f\": rpc error: code = NotFound desc = could not find container \"1ccefcec6533ab4fae508e219a0deeab0267968c40073b522d3bc42490e2c40f\": container with ID starting with 1ccefcec6533ab4fae508e219a0deeab0267968c40073b522d3bc42490e2c40f not found: ID does not exist"
Sep 10 13:24:01.810785 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.810757    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66944f8875-fc8l9"]
Sep 10 13:24:01.818272 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:01.818252    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66944f8875-fc8l9"]
Sep 10 13:24:03.713123 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:03.713094    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa5dde79-310f-4442-a2ac-f7693c46b3ef" path="/var/lib/kubelet/pods/aa5dde79-310f-4442-a2ac-f7693c46b3ef/volumes"
Sep 10 13:24:22.050250 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:22.050212    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:24:22.065234 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:22.065210    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:24:22.867056 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:22.867025    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:24:44.668372 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.668338    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cbf75fcbc-j6svp"]
Sep 10 13:24:44.668848 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.668681    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5dde79-310f-4442-a2ac-f7693c46b3ef" containerName="console"
Sep 10 13:24:44.668848 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.668693    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5dde79-310f-4442-a2ac-f7693c46b3ef" containerName="console"
Sep 10 13:24:44.668848 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.668741    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa5dde79-310f-4442-a2ac-f7693c46b3ef" containerName="console"
Sep 10 13:24:44.671188 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.671172    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.685509 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.685482    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cbf75fcbc-j6svp"]
Sep 10 13:24:44.762119 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.762084    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/83577162-25b2-427a-8f9e-ae6530121d97-console-serving-cert\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.762119 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.762121    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/83577162-25b2-427a-8f9e-ae6530121d97-console-oauth-config\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.762346 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.762212    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/83577162-25b2-427a-8f9e-ae6530121d97-console-config\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.762346 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.762235    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/83577162-25b2-427a-8f9e-ae6530121d97-oauth-serving-cert\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.762346 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.762253    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83577162-25b2-427a-8f9e-ae6530121d97-trusted-ca-bundle\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.762346 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.762276    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/83577162-25b2-427a-8f9e-ae6530121d97-service-ca\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.762346 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.762299    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-847kq\" (UniqueName: \"kubernetes.io/projected/83577162-25b2-427a-8f9e-ae6530121d97-kube-api-access-847kq\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.863011 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.862976    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/83577162-25b2-427a-8f9e-ae6530121d97-console-config\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.863011 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.863011    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/83577162-25b2-427a-8f9e-ae6530121d97-oauth-serving-cert\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.863228 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.863029    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83577162-25b2-427a-8f9e-ae6530121d97-trusted-ca-bundle\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.863228 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.863046    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/83577162-25b2-427a-8f9e-ae6530121d97-service-ca\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.863228 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.863065    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-847kq\" (UniqueName: \"kubernetes.io/projected/83577162-25b2-427a-8f9e-ae6530121d97-kube-api-access-847kq\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.863228 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.863083    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/83577162-25b2-427a-8f9e-ae6530121d97-console-serving-cert\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.863228 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.863116    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/83577162-25b2-427a-8f9e-ae6530121d97-console-oauth-config\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.863851 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.863821    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/83577162-25b2-427a-8f9e-ae6530121d97-oauth-serving-cert\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.863949 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.863860    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/83577162-25b2-427a-8f9e-ae6530121d97-service-ca\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.864094 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.863954    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/83577162-25b2-427a-8f9e-ae6530121d97-console-config\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.864094 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.863997    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83577162-25b2-427a-8f9e-ae6530121d97-trusted-ca-bundle\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.865539 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.865517    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/83577162-25b2-427a-8f9e-ae6530121d97-console-serving-cert\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.865668 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.865651    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/83577162-25b2-427a-8f9e-ae6530121d97-console-oauth-config\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.872321 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.872301    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-847kq\" (UniqueName: \"kubernetes.io/projected/83577162-25b2-427a-8f9e-ae6530121d97-kube-api-access-847kq\") pod \"console-6cbf75fcbc-j6svp\" (UID: \"83577162-25b2-427a-8f9e-ae6530121d97\") " pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:44.980296 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:44.980259    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:45.254495 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:45.254414    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cbf75fcbc-j6svp"]
Sep 10 13:24:45.258169 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:24:45.258132    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83577162_25b2_427a_8f9e_ae6530121d97.slice/crio-2b1a53e51b558dc034263b0c069dbd35b2cc5a1aa96dc1188306ac6473779173 WatchSource:0}: Error finding container 2b1a53e51b558dc034263b0c069dbd35b2cc5a1aa96dc1188306ac6473779173: Status 404 returned error can't find the container with id 2b1a53e51b558dc034263b0c069dbd35b2cc5a1aa96dc1188306ac6473779173
Sep 10 13:24:45.936546 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:45.936510    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cbf75fcbc-j6svp" event={"ID":"83577162-25b2-427a-8f9e-ae6530121d97","Type":"ContainerStarted","Data":"fa276808a9debbae924da4b869453386d600c41e5b32cdbff3d40e42778c05f3"}
Sep 10 13:24:45.936975 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:45.936567    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cbf75fcbc-j6svp" event={"ID":"83577162-25b2-427a-8f9e-ae6530121d97","Type":"ContainerStarted","Data":"2b1a53e51b558dc034263b0c069dbd35b2cc5a1aa96dc1188306ac6473779173"}
Sep 10 13:24:45.960503 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:45.960459    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cbf75fcbc-j6svp" podStartSLOduration=1.960445758 podStartE2EDuration="1.960445758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:24:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:24:45.958802257 +0000 UTC m=+396.777840657" watchObservedRunningTime="2026-09-10 13:24:45.960445758 +0000 UTC m=+396.779484159"
Sep 10 13:24:54.980659 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:54.980625    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:54.981034 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:54.980709    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:54.985502 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:54.985480    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:55.974647 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:55.974621    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cbf75fcbc-j6svp"
Sep 10 13:24:56.025382 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:24:56.025351    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66d45ddcb5-z4xr2"]
Sep 10 13:25:21.046934 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.046878    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66d45ddcb5-z4xr2" podUID="d5d84b92-63b2-41ad-b6c1-9edaaa494e3b" containerName="console" containerID="cri-o://5d4132df0a2c381d9714e9e7e950821a688183ad1b67aff7e2751b0ef2e69932" gracePeriod=15
Sep 10 13:25:21.289986 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.289965    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:25:21.342478 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.342410    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-trusted-ca-bundle\") pod \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") "
Sep 10 13:25:21.342478 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.342453    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jskss\" (UniqueName: \"kubernetes.io/projected/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-kube-api-access-jskss\") pod \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") "
Sep 10 13:25:21.342649 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.342535    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-config\") pod \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") "
Sep 10 13:25:21.342649 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.342592    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-serving-cert\") pod \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") "
Sep 10 13:25:21.342649 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.342637    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-oauth-serving-cert\") pod \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") "
Sep 10 13:25:21.342805 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.342664    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-service-ca\") pod \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") "
Sep 10 13:25:21.342805 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.342694    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-oauth-config\") pod \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\" (UID: \"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b\") "
Sep 10 13:25:21.342913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.342816    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d5d84b92-63b2-41ad-b6c1-9edaaa494e3b" (UID: "d5d84b92-63b2-41ad-b6c1-9edaaa494e3b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:25:21.343094 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.343068    2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-trusted-ca-bundle\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:25:21.343228 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.343106    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d5d84b92-63b2-41ad-b6c1-9edaaa494e3b" (UID: "d5d84b92-63b2-41ad-b6c1-9edaaa494e3b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:25:21.343228 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.343136    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-config" (OuterVolumeSpecName: "console-config") pod "d5d84b92-63b2-41ad-b6c1-9edaaa494e3b" (UID: "d5d84b92-63b2-41ad-b6c1-9edaaa494e3b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:25:21.343228 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.343196    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-service-ca" (OuterVolumeSpecName: "service-ca") pod "d5d84b92-63b2-41ad-b6c1-9edaaa494e3b" (UID: "d5d84b92-63b2-41ad-b6c1-9edaaa494e3b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:25:21.344548 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.344521    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-kube-api-access-jskss" (OuterVolumeSpecName: "kube-api-access-jskss") pod "d5d84b92-63b2-41ad-b6c1-9edaaa494e3b" (UID: "d5d84b92-63b2-41ad-b6c1-9edaaa494e3b"). InnerVolumeSpecName "kube-api-access-jskss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:25:21.344548 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.344540    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d5d84b92-63b2-41ad-b6c1-9edaaa494e3b" (UID: "d5d84b92-63b2-41ad-b6c1-9edaaa494e3b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:25:21.344696 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.344682    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d5d84b92-63b2-41ad-b6c1-9edaaa494e3b" (UID: "d5d84b92-63b2-41ad-b6c1-9edaaa494e3b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:25:21.444325 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.444300    2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-oauth-serving-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:25:21.444325 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.444320    2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:25:21.444459 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.444331    2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-oauth-config\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:25:21.444459 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.444339    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jskss\" (UniqueName: \"kubernetes.io/projected/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-kube-api-access-jskss\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:25:21.444459 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.444348    2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-config\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:25:21.444459 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:21.444356    2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b-console-serving-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:25:22.065238 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:22.065207    2551 generic.go:356] "Generic (PLEG): container finished" podID="d5d84b92-63b2-41ad-b6c1-9edaaa494e3b" containerID="5d4132df0a2c381d9714e9e7e950821a688183ad1b67aff7e2751b0ef2e69932" exitCode=0
Sep 10 13:25:22.065619 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:22.065252    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d45ddcb5-z4xr2"
Sep 10 13:25:22.065619 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:22.065280    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d45ddcb5-z4xr2" event={"ID":"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b","Type":"ContainerDied","Data":"5d4132df0a2c381d9714e9e7e950821a688183ad1b67aff7e2751b0ef2e69932"}
Sep 10 13:25:22.065619 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:22.065330    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d45ddcb5-z4xr2" event={"ID":"d5d84b92-63b2-41ad-b6c1-9edaaa494e3b","Type":"ContainerDied","Data":"b049743dbd52a6981b85358b5b385dbf712727a501200112019edd60d4e26a18"}
Sep 10 13:25:22.065619 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:22.065352    2551 scope.go:117] "RemoveContainer" containerID="5d4132df0a2c381d9714e9e7e950821a688183ad1b67aff7e2751b0ef2e69932"
Sep 10 13:25:22.072879 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:22.072863    2551 scope.go:117] "RemoveContainer" containerID="5d4132df0a2c381d9714e9e7e950821a688183ad1b67aff7e2751b0ef2e69932"
Sep 10 13:25:22.073127 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:25:22.073112    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d4132df0a2c381d9714e9e7e950821a688183ad1b67aff7e2751b0ef2e69932\": container with ID starting with 5d4132df0a2c381d9714e9e7e950821a688183ad1b67aff7e2751b0ef2e69932 not found: ID does not exist" containerID="5d4132df0a2c381d9714e9e7e950821a688183ad1b67aff7e2751b0ef2e69932"
Sep 10 13:25:22.073185 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:22.073132    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d4132df0a2c381d9714e9e7e950821a688183ad1b67aff7e2751b0ef2e69932"} err="failed to get container status \"5d4132df0a2c381d9714e9e7e950821a688183ad1b67aff7e2751b0ef2e69932\": rpc error: code = NotFound desc = could not find container \"5d4132df0a2c381d9714e9e7e950821a688183ad1b67aff7e2751b0ef2e69932\": container with ID starting with 5d4132df0a2c381d9714e9e7e950821a688183ad1b67aff7e2751b0ef2e69932 not found: ID does not exist"
Sep 10 13:25:22.084789 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:22.084766    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66d45ddcb5-z4xr2"]
Sep 10 13:25:22.089170 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:22.089151    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66d45ddcb5-z4xr2"]
Sep 10 13:25:23.708455 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:25:23.708422    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5d84b92-63b2-41ad-b6c1-9edaaa494e3b" path="/var/lib/kubelet/pods/d5d84b92-63b2-41ad-b6c1-9edaaa494e3b/volumes"
Sep 10 13:26:06.497387 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.497350    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-d82ld"]
Sep 10 13:26:06.497872 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.497644    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5d84b92-63b2-41ad-b6c1-9edaaa494e3b" containerName="console"
Sep 10 13:26:06.497872 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.497656    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5d84b92-63b2-41ad-b6c1-9edaaa494e3b" containerName="console"
Sep 10 13:26:06.497872 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.497712    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5d84b92-63b2-41ad-b6c1-9edaaa494e3b" containerName="console"
Sep 10 13:26:06.500454 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.500439    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-d82ld"
Sep 10 13:26:06.504384 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.504364    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 13:26:06.504520 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.504405    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 13:26:06.505470 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.505448    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-cp4d4\""
Sep 10 13:26:06.513480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.513458    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-d82ld"]
Sep 10 13:26:06.586229 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.586202    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thq8v\" (UniqueName: \"kubernetes.io/projected/a1d325a2-4779-48d9-8af0-e6e2e06bb690-kube-api-access-thq8v\") pod \"cert-manager-webhook-6486ffd997-d82ld\" (UID: \"a1d325a2-4779-48d9-8af0-e6e2e06bb690\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d82ld"
Sep 10 13:26:06.586353 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.586253    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1d325a2-4779-48d9-8af0-e6e2e06bb690-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-d82ld\" (UID: \"a1d325a2-4779-48d9-8af0-e6e2e06bb690\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d82ld"
Sep 10 13:26:06.686645 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.686613    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1d325a2-4779-48d9-8af0-e6e2e06bb690-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-d82ld\" (UID: \"a1d325a2-4779-48d9-8af0-e6e2e06bb690\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d82ld"
Sep 10 13:26:06.686772 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.686682    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thq8v\" (UniqueName: \"kubernetes.io/projected/a1d325a2-4779-48d9-8af0-e6e2e06bb690-kube-api-access-thq8v\") pod \"cert-manager-webhook-6486ffd997-d82ld\" (UID: \"a1d325a2-4779-48d9-8af0-e6e2e06bb690\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d82ld"
Sep 10 13:26:06.708278 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.708214    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1d325a2-4779-48d9-8af0-e6e2e06bb690-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-d82ld\" (UID: \"a1d325a2-4779-48d9-8af0-e6e2e06bb690\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d82ld"
Sep 10 13:26:06.710611 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.710565    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thq8v\" (UniqueName: \"kubernetes.io/projected/a1d325a2-4779-48d9-8af0-e6e2e06bb690-kube-api-access-thq8v\") pod \"cert-manager-webhook-6486ffd997-d82ld\" (UID: \"a1d325a2-4779-48d9-8af0-e6e2e06bb690\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d82ld"
Sep 10 13:26:06.823062 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:06.822986    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-d82ld"
Sep 10 13:26:07.113096 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:07.113025    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-d82ld"]
Sep 10 13:26:07.116030 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:26:07.116004    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1d325a2_4779_48d9_8af0_e6e2e06bb690.slice/crio-e845378f4686461fa3968eb5959886dae647d1f17da7bd74ab08496f06d2cfeb WatchSource:0}: Error finding container e845378f4686461fa3968eb5959886dae647d1f17da7bd74ab08496f06d2cfeb: Status 404 returned error can't find the container with id e845378f4686461fa3968eb5959886dae647d1f17da7bd74ab08496f06d2cfeb
Sep 10 13:26:07.228167 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:07.228138    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-d82ld" event={"ID":"a1d325a2-4779-48d9-8af0-e6e2e06bb690","Type":"ContainerStarted","Data":"e845378f4686461fa3968eb5959886dae647d1f17da7bd74ab08496f06d2cfeb"}
Sep 10 13:26:12.251281 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:12.251248    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-d82ld" event={"ID":"a1d325a2-4779-48d9-8af0-e6e2e06bb690","Type":"ContainerStarted","Data":"5bc75a6347b5c6cf32586a92b51a2f34154c1a6baa21c0ca94051840385a2cdd"}
Sep 10 13:26:12.251745 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:12.251302    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-d82ld"
Sep 10 13:26:12.275756 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:12.275709    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-d82ld" podStartSLOduration=2.013296526 podStartE2EDuration="6.27569472s" totalImagesPullingTime="4.262398194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:26:12.274070606 +0000 UTC m=+483.093109008" watchObservedRunningTime="2026-09-10 13:26:12.27569472 +0000 UTC m=+483.094733120"
Sep 10 13:26:18.258127 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:18.258096    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-d82ld"
Sep 10 13:26:23.333632 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:23.333599    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rmgbv"]
Sep 10 13:26:23.336988 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:23.336969    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rmgbv"
Sep 10 13:26:23.343356 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:23.343114    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-7qzqh\""
Sep 10 13:26:23.344004 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:23.343989    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 13:26:23.345302 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:23.345285    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 13:26:23.358364 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:23.358343    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rmgbv"]
Sep 10 13:26:23.435883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:23.435848    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c977b95-db5d-46c8-8b51-ea95c64e9790-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rmgbv\" (UID: \"4c977b95-db5d-46c8-8b51-ea95c64e9790\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rmgbv"
Sep 10 13:26:23.436054 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:23.435892    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqwt5\" (UniqueName: \"kubernetes.io/projected/4c977b95-db5d-46c8-8b51-ea95c64e9790-kube-api-access-qqwt5\") pod \"openshift-lws-operator-fd8ccff4c-rmgbv\" (UID: \"4c977b95-db5d-46c8-8b51-ea95c64e9790\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rmgbv"
Sep 10 13:26:23.536192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:23.536157    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c977b95-db5d-46c8-8b51-ea95c64e9790-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rmgbv\" (UID: \"4c977b95-db5d-46c8-8b51-ea95c64e9790\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rmgbv"
Sep 10 13:26:23.536297 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:23.536210    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqwt5\" (UniqueName: \"kubernetes.io/projected/4c977b95-db5d-46c8-8b51-ea95c64e9790-kube-api-access-qqwt5\") pod \"openshift-lws-operator-fd8ccff4c-rmgbv\" (UID: \"4c977b95-db5d-46c8-8b51-ea95c64e9790\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rmgbv"
Sep 10 13:26:23.536525 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:23.536507    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c977b95-db5d-46c8-8b51-ea95c64e9790-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rmgbv\" (UID: \"4c977b95-db5d-46c8-8b51-ea95c64e9790\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rmgbv"
Sep 10 13:26:23.560791 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:23.560763    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqwt5\" (UniqueName: \"kubernetes.io/projected/4c977b95-db5d-46c8-8b51-ea95c64e9790-kube-api-access-qqwt5\") pod \"openshift-lws-operator-fd8ccff4c-rmgbv\" (UID: \"4c977b95-db5d-46c8-8b51-ea95c64e9790\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rmgbv"
Sep 10 13:26:23.645952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:23.645883    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rmgbv"
Sep 10 13:26:23.943884 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:23.943770    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rmgbv"]
Sep 10 13:26:23.947106 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:26:23.947081    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c977b95_db5d_46c8_8b51_ea95c64e9790.slice/crio-6bcb2e3f7950f7b222b8330b542190923f299e072a516c36f2373c9e0c24e90b WatchSource:0}: Error finding container 6bcb2e3f7950f7b222b8330b542190923f299e072a516c36f2373c9e0c24e90b: Status 404 returned error can't find the container with id 6bcb2e3f7950f7b222b8330b542190923f299e072a516c36f2373c9e0c24e90b
Sep 10 13:26:24.294369 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:24.294334    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rmgbv" event={"ID":"4c977b95-db5d-46c8-8b51-ea95c64e9790","Type":"ContainerStarted","Data":"6bcb2e3f7950f7b222b8330b542190923f299e072a516c36f2373c9e0c24e90b"}
Sep 10 13:26:27.310243 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:27.310206    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rmgbv" event={"ID":"4c977b95-db5d-46c8-8b51-ea95c64e9790","Type":"ContainerStarted","Data":"afae341541e754cd51e1df8a98159a3c8c937b91431f1f6b79d6906a1ea068a1"}
Sep 10 13:26:27.330388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:27.330333    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rmgbv" podStartSLOduration=1.7830606260000001 podStartE2EDuration="4.330316084s" totalImagesPullingTime="2.547255458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:26:27.328278024 +0000 UTC m=+498.147316421" watchObservedRunningTime="2026-09-10 13:26:27.330316084 +0000 UTC m=+498.149354483"
Sep 10 13:26:43.547870 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.547833    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"]
Sep 10 13:26:43.590239 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.590207    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"]
Sep 10 13:26:43.590392 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.590331    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.594977 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.594959    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-m6d87\""
Sep 10 13:26:43.595205 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.595187    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 13:26:43.595205 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.595199    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 13:26:43.595355 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.595183    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 13:26:43.595355 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.595189    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 13:26:43.704382 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.704351    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.704382 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.704386    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-manifests\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.704676 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.704438    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-tmp\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.704676 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.704489    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-webhook-cert\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.704676 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.704610    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zfqk\" (UniqueName: \"kubernetes.io/projected/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-kube-api-access-4zfqk\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.805525 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.805439    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.805525 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.805482    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-manifests\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.805777 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.805681    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-tmp\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.805777 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.805774    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-webhook-cert\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.805894 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.805812    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zfqk\" (UniqueName: \"kubernetes.io/projected/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-kube-api-access-4zfqk\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.805950 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.805907    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-manifests\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.807859 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.807829    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-tmp\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.808129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.808110    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-webhook-cert\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.808198 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.808120    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.816639 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.816620    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zfqk\" (UniqueName: \"kubernetes.io/projected/ae416f04-8a04-4e64-9e25-aa8bdacdc13b-kube-api-access-4zfqk\") pod \"opendatahub-operator-controller-manager-84d444f55-8mtls\" (UID: \"ae416f04-8a04-4e64-9e25-aa8bdacdc13b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:43.899790 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:43.899758    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:44.203133 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:44.203110    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"]
Sep 10 13:26:44.205331 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:26:44.205302    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae416f04_8a04_4e64_9e25_aa8bdacdc13b.slice/crio-6b914eb6fe5911db8cc7b210b0cfea45bba11c2d303b367a26112e4a2c037e5d WatchSource:0}: Error finding container 6b914eb6fe5911db8cc7b210b0cfea45bba11c2d303b367a26112e4a2c037e5d: Status 404 returned error can't find the container with id 6b914eb6fe5911db8cc7b210b0cfea45bba11c2d303b367a26112e4a2c037e5d
Sep 10 13:26:44.374532 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:44.374497    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls" event={"ID":"ae416f04-8a04-4e64-9e25-aa8bdacdc13b","Type":"ContainerStarted","Data":"6b914eb6fe5911db8cc7b210b0cfea45bba11c2d303b367a26112e4a2c037e5d"}
Sep 10 13:26:47.392606 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:47.392561    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls" event={"ID":"ae416f04-8a04-4e64-9e25-aa8bdacdc13b","Type":"ContainerStarted","Data":"a4b9af7bd4282e49cde50e279b9c905d7796dc127762f4e591ce51f1476c9b50"}
Sep 10 13:26:48.397865 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:48.397833    2551 generic.go:356] "Generic (PLEG): container finished" podID="ae416f04-8a04-4e64-9e25-aa8bdacdc13b" containerID="a4b9af7bd4282e49cde50e279b9c905d7796dc127762f4e591ce51f1476c9b50" exitCode=0
Sep 10 13:26:48.398227 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:48.397918    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls" event={"ID":"ae416f04-8a04-4e64-9e25-aa8bdacdc13b","Type":"ContainerDied","Data":"a4b9af7bd4282e49cde50e279b9c905d7796dc127762f4e591ce51f1476c9b50"}
Sep 10 13:26:49.404427 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:49.404395    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls" event={"ID":"ae416f04-8a04-4e64-9e25-aa8bdacdc13b","Type":"ContainerStarted","Data":"337b7c11786b772f5f92407a09c867196e9fa4d771cb7014fe2d35b716413e22"}
Sep 10 13:26:49.405003 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:49.404523    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:26:49.430148 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:26:49.430097    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls" podStartSLOduration=2.01350283 podStartE2EDuration="6.430082882s" totalImagesPullingTime="3.385385267s" totalInitContainerRuntime="1.031194785s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:26:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:26:49.429789213 +0000 UTC m=+520.248827615" watchObservedRunningTime="2026-09-10 13:26:49.430082882 +0000 UTC m=+520.249121282"
Sep 10 13:27:00.411169 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:00.410636    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-8mtls"
Sep 10 13:27:02.726385 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.726313    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"]
Sep 10 13:27:02.735277 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.735257    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:02.737681 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.737660    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 13:27:02.738959 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.738916    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-v4qfv\""
Sep 10 13:27:02.739047 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.738916    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 13:27:02.739047 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.738980    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 13:27:02.746759 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.746736    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"]
Sep 10 13:27:02.870949 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.870922    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d18d8f64-cfa1-4e94-b938-b00fd21cc581-manager-config\") pod \"lws-controller-manager-6746899c78-fgqq8\" (UID: \"d18d8f64-cfa1-4e94-b938-b00fd21cc581\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:02.871126 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.870956    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d18d8f64-cfa1-4e94-b938-b00fd21cc581-metrics-cert\") pod \"lws-controller-manager-6746899c78-fgqq8\" (UID: \"d18d8f64-cfa1-4e94-b938-b00fd21cc581\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:02.871126 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.871049    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d18d8f64-cfa1-4e94-b938-b00fd21cc581-cert\") pod \"lws-controller-manager-6746899c78-fgqq8\" (UID: \"d18d8f64-cfa1-4e94-b938-b00fd21cc581\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:02.871265 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.871234    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m9b4\" (UniqueName: \"kubernetes.io/projected/d18d8f64-cfa1-4e94-b938-b00fd21cc581-kube-api-access-5m9b4\") pod \"lws-controller-manager-6746899c78-fgqq8\" (UID: \"d18d8f64-cfa1-4e94-b938-b00fd21cc581\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:02.972095 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.972052    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d18d8f64-cfa1-4e94-b938-b00fd21cc581-manager-config\") pod \"lws-controller-manager-6746899c78-fgqq8\" (UID: \"d18d8f64-cfa1-4e94-b938-b00fd21cc581\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:02.972095 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.972095    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d18d8f64-cfa1-4e94-b938-b00fd21cc581-metrics-cert\") pod \"lws-controller-manager-6746899c78-fgqq8\" (UID: \"d18d8f64-cfa1-4e94-b938-b00fd21cc581\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:02.972270 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.972125    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d18d8f64-cfa1-4e94-b938-b00fd21cc581-cert\") pod \"lws-controller-manager-6746899c78-fgqq8\" (UID: \"d18d8f64-cfa1-4e94-b938-b00fd21cc581\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:02.972270 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.972165    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m9b4\" (UniqueName: \"kubernetes.io/projected/d18d8f64-cfa1-4e94-b938-b00fd21cc581-kube-api-access-5m9b4\") pod \"lws-controller-manager-6746899c78-fgqq8\" (UID: \"d18d8f64-cfa1-4e94-b938-b00fd21cc581\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:02.972706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.972676    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d18d8f64-cfa1-4e94-b938-b00fd21cc581-manager-config\") pod \"lws-controller-manager-6746899c78-fgqq8\" (UID: \"d18d8f64-cfa1-4e94-b938-b00fd21cc581\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:02.974382 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.974351    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d18d8f64-cfa1-4e94-b938-b00fd21cc581-cert\") pod \"lws-controller-manager-6746899c78-fgqq8\" (UID: \"d18d8f64-cfa1-4e94-b938-b00fd21cc581\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:02.974505 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.974478    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d18d8f64-cfa1-4e94-b938-b00fd21cc581-metrics-cert\") pod \"lws-controller-manager-6746899c78-fgqq8\" (UID: \"d18d8f64-cfa1-4e94-b938-b00fd21cc581\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:02.987152 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:02.987064    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m9b4\" (UniqueName: \"kubernetes.io/projected/d18d8f64-cfa1-4e94-b938-b00fd21cc581-kube-api-access-5m9b4\") pod \"lws-controller-manager-6746899c78-fgqq8\" (UID: \"d18d8f64-cfa1-4e94-b938-b00fd21cc581\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:03.044731 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.044693    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:03.329810 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.329731    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"]
Sep 10 13:27:03.333298 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:27:03.333269    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd18d8f64_cfa1_4e94_b938_b00fd21cc581.slice/crio-908819a120fecb3a497f53514a5483abd92d29a7cfd56e328cf3252a0d57ae85 WatchSource:0}: Error finding container 908819a120fecb3a497f53514a5483abd92d29a7cfd56e328cf3252a0d57ae85: Status 404 returned error can't find the container with id 908819a120fecb3a497f53514a5483abd92d29a7cfd56e328cf3252a0d57ae85
Sep 10 13:27:03.467596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.467542    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8" event={"ID":"d18d8f64-cfa1-4e94-b938-b00fd21cc581","Type":"ContainerStarted","Data":"908819a120fecb3a497f53514a5483abd92d29a7cfd56e328cf3252a0d57ae85"}
Sep 10 13:27:03.637324 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.637239    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4"]
Sep 10 13:27:03.639361 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.639338    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4"
Sep 10 13:27:03.645490 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.645469    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 13:27:03.645628 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.645512    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 13:27:03.647082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.646948    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 13:27:03.647082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.646794    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-rmg4s\""
Sep 10 13:27:03.647082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.647057    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 13:27:03.666786 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.666760    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4"]
Sep 10 13:27:03.778367 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.778336    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7jrb\" (UniqueName: \"kubernetes.io/projected/89ee133c-2fe4-4ee1-8638-41fe4bc24fcd-kube-api-access-n7jrb\") pod \"kube-auth-proxy-cb8b5959f-p7pl4\" (UID: \"89ee133c-2fe4-4ee1-8638-41fe4bc24fcd\") " pod="openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4"
Sep 10 13:27:03.778749 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.778392    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89ee133c-2fe4-4ee1-8638-41fe4bc24fcd-tmp\") pod \"kube-auth-proxy-cb8b5959f-p7pl4\" (UID: \"89ee133c-2fe4-4ee1-8638-41fe4bc24fcd\") " pod="openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4"
Sep 10 13:27:03.778749 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.778491    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89ee133c-2fe4-4ee1-8638-41fe4bc24fcd-tls-certs\") pod \"kube-auth-proxy-cb8b5959f-p7pl4\" (UID: \"89ee133c-2fe4-4ee1-8638-41fe4bc24fcd\") " pod="openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4"
Sep 10 13:27:03.879394 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.879369    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89ee133c-2fe4-4ee1-8638-41fe4bc24fcd-tls-certs\") pod \"kube-auth-proxy-cb8b5959f-p7pl4\" (UID: \"89ee133c-2fe4-4ee1-8638-41fe4bc24fcd\") " pod="openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4"
Sep 10 13:27:03.879554 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.879411    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7jrb\" (UniqueName: \"kubernetes.io/projected/89ee133c-2fe4-4ee1-8638-41fe4bc24fcd-kube-api-access-n7jrb\") pod \"kube-auth-proxy-cb8b5959f-p7pl4\" (UID: \"89ee133c-2fe4-4ee1-8638-41fe4bc24fcd\") " pod="openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4"
Sep 10 13:27:03.879554 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.879445    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89ee133c-2fe4-4ee1-8638-41fe4bc24fcd-tmp\") pod \"kube-auth-proxy-cb8b5959f-p7pl4\" (UID: \"89ee133c-2fe4-4ee1-8638-41fe4bc24fcd\") " pod="openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4"
Sep 10 13:27:03.881628 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.881605    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89ee133c-2fe4-4ee1-8638-41fe4bc24fcd-tmp\") pod \"kube-auth-proxy-cb8b5959f-p7pl4\" (UID: \"89ee133c-2fe4-4ee1-8638-41fe4bc24fcd\") " pod="openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4"
Sep 10 13:27:03.881724 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.881706    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89ee133c-2fe4-4ee1-8638-41fe4bc24fcd-tls-certs\") pod \"kube-auth-proxy-cb8b5959f-p7pl4\" (UID: \"89ee133c-2fe4-4ee1-8638-41fe4bc24fcd\") " pod="openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4"
Sep 10 13:27:03.889150 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.889104    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7jrb\" (UniqueName: \"kubernetes.io/projected/89ee133c-2fe4-4ee1-8638-41fe4bc24fcd-kube-api-access-n7jrb\") pod \"kube-auth-proxy-cb8b5959f-p7pl4\" (UID: \"89ee133c-2fe4-4ee1-8638-41fe4bc24fcd\") " pod="openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4"
Sep 10 13:27:03.950059 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:03.950037    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4"
Sep 10 13:27:04.226224 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:04.226077    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4"]
Sep 10 13:27:04.228587 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:27:04.228547    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ee133c_2fe4_4ee1_8638_41fe4bc24fcd.slice/crio-ae807d1baa2a8b14aec72b14ff1f9f80b54647cec9ace14be64b19e6b1522700 WatchSource:0}: Error finding container ae807d1baa2a8b14aec72b14ff1f9f80b54647cec9ace14be64b19e6b1522700: Status 404 returned error can't find the container with id ae807d1baa2a8b14aec72b14ff1f9f80b54647cec9ace14be64b19e6b1522700
Sep 10 13:27:04.474672 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:04.474630    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4" event={"ID":"89ee133c-2fe4-4ee1-8638-41fe4bc24fcd","Type":"ContainerStarted","Data":"ae807d1baa2a8b14aec72b14ff1f9f80b54647cec9ace14be64b19e6b1522700"}
Sep 10 13:27:05.481656 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:05.481613    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8" event={"ID":"d18d8f64-cfa1-4e94-b938-b00fd21cc581","Type":"ContainerStarted","Data":"623a429409d034c04008ce47def905b7ddb03681ba03714892eb234b57a58b3a"}
Sep 10 13:27:05.482156 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:05.481726    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:27:05.507175 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:05.507107    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8" podStartSLOduration=1.547173474 podStartE2EDuration="3.507087057s" totalImagesPullingTime="1.959913583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:27:05.506293831 +0000 UTC m=+536.325332234" watchObservedRunningTime="2026-09-10 13:27:05.507087057 +0000 UTC m=+536.326125457"
Sep 10 13:27:07.492126 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:07.492089    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4" event={"ID":"89ee133c-2fe4-4ee1-8638-41fe4bc24fcd","Type":"ContainerStarted","Data":"549a6ffcbe04ebff3b7e9f37d992b7ee914e42112a1ddce52c5ef50d4849bceb"}
Sep 10 13:27:07.512459 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:07.512408    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-cb8b5959f-p7pl4" podStartSLOduration=1.619706936 podStartE2EDuration="4.512387945s" totalImagesPullingTime="2.892681009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:27:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:27:07.511454312 +0000 UTC m=+538.330492712" watchObservedRunningTime="2026-09-10 13:27:07.512387945 +0000 UTC m=+538.331426346"
Sep 10 13:27:16.488911 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:27:16.488881    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6746899c78-fgqq8"
Sep 10 13:28:09.615788 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:28:09.615760    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:28:09.617823 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:28:09.617803    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:29:35.361098 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.361062    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"]
Sep 10 13:29:35.363325 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.363308    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.379997 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.379975    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 13:29:35.380292 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.380272    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.380340 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.380301    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.380340 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.380322    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36051b6e-ece1-419c-9a57-3610df345db8-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.380421 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.380342    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36051b6e-ece1-419c-9a57-3610df345db8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.380421 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.380394    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36051b6e-ece1-419c-9a57-3610df345db8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.380539 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.380432    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.380539 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.380499    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz65j\" (UniqueName: \"kubernetes.io/projected/36051b6e-ece1-419c-9a57-3610df345db8-kube-api-access-bz65j\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.380677 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.380543    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.380677 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.380601    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.385883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.385866    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-x55p4\""
Sep 10 13:29:35.402926 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.402899    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"]
Sep 10 13:29:35.481816 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.481791    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.481944 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.481822    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.481944 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.481848    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36051b6e-ece1-419c-9a57-3610df345db8-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.481944 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.481875    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36051b6e-ece1-419c-9a57-3610df345db8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.481944 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.481898    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36051b6e-ece1-419c-9a57-3610df345db8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.481944 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.481925    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.482210 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.481950    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz65j\" (UniqueName: \"kubernetes.io/projected/36051b6e-ece1-419c-9a57-3610df345db8-kube-api-access-bz65j\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.482210 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.481979    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.482210 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.482012    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.482408 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.482382    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.482526 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.482459    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.482526 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.482496    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.482644 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.482538    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.482715 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.482694    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36051b6e-ece1-419c-9a57-3610df345db8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.484087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.484067    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36051b6e-ece1-419c-9a57-3610df345db8-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.484460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.484440    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36051b6e-ece1-419c-9a57-3610df345db8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.496789 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.496764    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz65j\" (UniqueName: \"kubernetes.io/projected/36051b6e-ece1-419c-9a57-3610df345db8-kube-api-access-bz65j\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.497879 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.497859    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36051b6e-ece1-419c-9a57-3610df345db8-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jhkw6\" (UID: \"36051b6e-ece1-419c-9a57-3610df345db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.672824 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.672737    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:35.951180 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.951018    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"]
Sep 10 13:29:35.953696 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:29:35.953673    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36051b6e_ece1_419c_9a57_3610df345db8.slice/crio-d513ad3e03c88a021cdb54e6ea54876e512982cda754d3c958dfc3cfb0b39d3e WatchSource:0}: Error finding container d513ad3e03c88a021cdb54e6ea54876e512982cda754d3c958dfc3cfb0b39d3e: Status 404 returned error can't find the container with id d513ad3e03c88a021cdb54e6ea54876e512982cda754d3c958dfc3cfb0b39d3e
Sep 10 13:29:35.955674 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:35.955646    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 13:29:36.096377 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:36.096348    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6" event={"ID":"36051b6e-ece1-419c-9a57-3610df345db8","Type":"ContainerStarted","Data":"d513ad3e03c88a021cdb54e6ea54876e512982cda754d3c958dfc3cfb0b39d3e"}
Sep 10 13:29:38.816534 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:38.816493    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:29:38.816781 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:38.816591    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:29:38.816781 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:38.816624    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:29:39.112666 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:39.112583    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6" event={"ID":"36051b6e-ece1-419c-9a57-3610df345db8","Type":"ContainerStarted","Data":"15958851f2c2395ac1bed91b3f8d90fa846ee103b9e09ffe541304af7c17f0c4"}
Sep 10 13:29:39.139345 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:39.139296    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6" podStartSLOduration=1.278862852 podStartE2EDuration="4.139281862s" totalImagesPullingTime="2.86041901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:29:39.137590144 +0000 UTC m=+689.956628536" watchObservedRunningTime="2026-09-10 13:29:39.139281862 +0000 UTC m=+689.958320262"
Sep 10 13:29:39.673098 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:39.673065    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:39.677500 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:39.677476    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:40.117406 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.117378    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:40.118381 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.118361    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:29:40.656634 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.656604    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:29:40.658945 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.658928    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"
Sep 10 13:29:40.662100 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.662076    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kpd5p\""
Sep 10 13:29:40.662100 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.662092    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 13:29:40.662270 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.662120    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 13:29:40.662270 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.662076    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 13:29:40.669528 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.669507    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:29:40.716058 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.716033    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:29:40.731100 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.731070    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpkq4\" (UniqueName: \"kubernetes.io/projected/8db4d60e-dff3-43c6-a241-022afc2147d9-kube-api-access-qpkq4\") pod \"limitador-limitador-84994f7ffc-5jzq6\" (UID: \"8db4d60e-dff3-43c6-a241-022afc2147d9\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"
Sep 10 13:29:40.731242 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.731219    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8db4d60e-dff3-43c6-a241-022afc2147d9-config-file\") pod \"limitador-limitador-84994f7ffc-5jzq6\" (UID: \"8db4d60e-dff3-43c6-a241-022afc2147d9\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"
Sep 10 13:29:40.832417 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.832391    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpkq4\" (UniqueName: \"kubernetes.io/projected/8db4d60e-dff3-43c6-a241-022afc2147d9-kube-api-access-qpkq4\") pod \"limitador-limitador-84994f7ffc-5jzq6\" (UID: \"8db4d60e-dff3-43c6-a241-022afc2147d9\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"
Sep 10 13:29:40.832543 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.832425    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8db4d60e-dff3-43c6-a241-022afc2147d9-config-file\") pod \"limitador-limitador-84994f7ffc-5jzq6\" (UID: \"8db4d60e-dff3-43c6-a241-022afc2147d9\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"
Sep 10 13:29:40.832991 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.832974    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8db4d60e-dff3-43c6-a241-022afc2147d9-config-file\") pod \"limitador-limitador-84994f7ffc-5jzq6\" (UID: \"8db4d60e-dff3-43c6-a241-022afc2147d9\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"
Sep 10 13:29:40.846029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.846002    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpkq4\" (UniqueName: \"kubernetes.io/projected/8db4d60e-dff3-43c6-a241-022afc2147d9-kube-api-access-qpkq4\") pod \"limitador-limitador-84994f7ffc-5jzq6\" (UID: \"8db4d60e-dff3-43c6-a241-022afc2147d9\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"
Sep 10 13:29:40.968702 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:40.968621    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"
Sep 10 13:29:41.143533 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.143497    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nmqfs"]
Sep 10 13:29:41.146048 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.146028    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-nmqfs"
Sep 10 13:29:41.150901 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.150882    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m2ck6\""
Sep 10 13:29:41.163952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.163933    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nmqfs"]
Sep 10 13:29:41.236405 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.236374    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwx5j\" (UniqueName: \"kubernetes.io/projected/1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf-kube-api-access-cwx5j\") pod \"authorino-7bcfd8f9cd-nmqfs\" (UID: \"1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf\") " pod="kuadrant-system/authorino-7bcfd8f9cd-nmqfs"
Sep 10 13:29:41.240389 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.240283    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:29:41.242954 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:29:41.242929    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db4d60e_dff3_43c6_a241_022afc2147d9.slice/crio-4802c2e36035d9cf7db33d5aa50d2c0c10fda3d02ba9e56a1c916bab4056016d WatchSource:0}: Error finding container 4802c2e36035d9cf7db33d5aa50d2c0c10fda3d02ba9e56a1c916bab4056016d: Status 404 returned error can't find the container with id 4802c2e36035d9cf7db33d5aa50d2c0c10fda3d02ba9e56a1c916bab4056016d
Sep 10 13:29:41.337378 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.337350    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwx5j\" (UniqueName: \"kubernetes.io/projected/1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf-kube-api-access-cwx5j\") pod \"authorino-7bcfd8f9cd-nmqfs\" (UID: \"1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf\") " pod="kuadrant-system/authorino-7bcfd8f9cd-nmqfs"
Sep 10 13:29:41.350717 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.350691    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwx5j\" (UniqueName: \"kubernetes.io/projected/1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf-kube-api-access-cwx5j\") pod \"authorino-7bcfd8f9cd-nmqfs\" (UID: \"1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf\") " pod="kuadrant-system/authorino-7bcfd8f9cd-nmqfs"
Sep 10 13:29:41.393463 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.393439    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-5wm8t"]
Sep 10 13:29:41.395686 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.395672    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-5wm8t"
Sep 10 13:29:41.410244 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.410224    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-5wm8t"]
Sep 10 13:29:41.438756 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.438732    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kh5c\" (UniqueName: \"kubernetes.io/projected/e884f0af-4f1f-43ca-abc3-95964e23c6a6-kube-api-access-6kh5c\") pod \"authorino-59c974bd48-5wm8t\" (UID: \"e884f0af-4f1f-43ca-abc3-95964e23c6a6\") " pod="kuadrant-system/authorino-59c974bd48-5wm8t"
Sep 10 13:29:41.455658 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.455639    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-nmqfs"
Sep 10 13:29:41.539252 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.539209    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kh5c\" (UniqueName: \"kubernetes.io/projected/e884f0af-4f1f-43ca-abc3-95964e23c6a6-kube-api-access-6kh5c\") pod \"authorino-59c974bd48-5wm8t\" (UID: \"e884f0af-4f1f-43ca-abc3-95964e23c6a6\") " pod="kuadrant-system/authorino-59c974bd48-5wm8t"
Sep 10 13:29:41.551907 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.551884    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kh5c\" (UniqueName: \"kubernetes.io/projected/e884f0af-4f1f-43ca-abc3-95964e23c6a6-kube-api-access-6kh5c\") pod \"authorino-59c974bd48-5wm8t\" (UID: \"e884f0af-4f1f-43ca-abc3-95964e23c6a6\") " pod="kuadrant-system/authorino-59c974bd48-5wm8t"
Sep 10 13:29:41.704424 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.704388    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-5wm8t"
Sep 10 13:29:41.731450 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.731336    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nmqfs"]
Sep 10 13:29:41.733806 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:29:41.733777    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e5cc9eb_31b6_4cb5_82a2_04b6cd82e5cf.slice/crio-b2b72e4191ac46d7660f3f795aa3cfbee5c9691312bafda1df939e356cac4bfe WatchSource:0}: Error finding container b2b72e4191ac46d7660f3f795aa3cfbee5c9691312bafda1df939e356cac4bfe: Status 404 returned error can't find the container with id b2b72e4191ac46d7660f3f795aa3cfbee5c9691312bafda1df939e356cac4bfe
Sep 10 13:29:41.981277 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:41.981242    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-5wm8t"]
Sep 10 13:29:41.982514 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:29:41.982481    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode884f0af_4f1f_43ca_abc3_95964e23c6a6.slice/crio-fd18b9049a5c9d5e428b529304982a1f09d06f202da92b243705442511c60850 WatchSource:0}: Error finding container fd18b9049a5c9d5e428b529304982a1f09d06f202da92b243705442511c60850: Status 404 returned error can't find the container with id fd18b9049a5c9d5e428b529304982a1f09d06f202da92b243705442511c60850
Sep 10 13:29:42.128615 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:42.128476    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-nmqfs" event={"ID":"1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf","Type":"ContainerStarted","Data":"b2b72e4191ac46d7660f3f795aa3cfbee5c9691312bafda1df939e356cac4bfe"}
Sep 10 13:29:42.130507 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:42.130473    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-5jzq6" event={"ID":"8db4d60e-dff3-43c6-a241-022afc2147d9","Type":"ContainerStarted","Data":"4802c2e36035d9cf7db33d5aa50d2c0c10fda3d02ba9e56a1c916bab4056016d"}
Sep 10 13:29:42.132499 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:42.132468    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-5wm8t" event={"ID":"e884f0af-4f1f-43ca-abc3-95964e23c6a6","Type":"ContainerStarted","Data":"fd18b9049a5c9d5e428b529304982a1f09d06f202da92b243705442511c60850"}
Sep 10 13:29:45.148373 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:45.148338    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-nmqfs" event={"ID":"1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf","Type":"ContainerStarted","Data":"d812366a52d968f80fc416cd973154ae1cd6959f6917d6124e0cda677bed18b0"}
Sep 10 13:29:45.149924 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:45.149901    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-5jzq6" event={"ID":"8db4d60e-dff3-43c6-a241-022afc2147d9","Type":"ContainerStarted","Data":"3e938dafd65559d63a5f593993a095a996e280d3563c6eba07302c40dc59d2cd"}
Sep 10 13:29:45.150049 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:45.149953    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"
Sep 10 13:29:45.151226 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:45.151208    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-5wm8t" event={"ID":"e884f0af-4f1f-43ca-abc3-95964e23c6a6","Type":"ContainerStarted","Data":"34ae02dfefb80b459f95a3b4caf4c1321dadf670bafbe9cbc458a85a2e7bd036"}
Sep 10 13:29:45.168949 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:45.168902    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-nmqfs" podStartSLOduration=1.7514790150000001 podStartE2EDuration="4.168887677s" totalImagesPullingTime="2.417408662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:29:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:29:45.167405199 +0000 UTC m=+695.986443599" watchObservedRunningTime="2026-09-10 13:29:45.168887677 +0000 UTC m=+695.987926077"
Sep 10 13:29:45.186313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:45.186258    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-5wm8t" podStartSLOduration=2.02725367 podStartE2EDuration="4.186243385s" totalImagesPullingTime="2.158989715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:29:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:29:45.184324018 +0000 UTC m=+696.003362421" watchObservedRunningTime="2026-09-10 13:29:45.186243385 +0000 UTC m=+696.005281834"
Sep 10 13:29:45.204811 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:45.204761    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-5jzq6" podStartSLOduration=2.307081598 podStartE2EDuration="5.204745493s" totalImagesPullingTime="2.897663895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:29:45.20307826 +0000 UTC m=+696.022116662" watchObservedRunningTime="2026-09-10 13:29:45.204745493 +0000 UTC m=+696.023783929"
Sep 10 13:29:45.221623 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:45.221595    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nmqfs"]
Sep 10 13:29:47.160198 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:47.160163    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-nmqfs" podUID="1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf" containerName="authorino" containerID="cri-o://d812366a52d968f80fc416cd973154ae1cd6959f6917d6124e0cda677bed18b0" gracePeriod=30
Sep 10 13:29:47.400735 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:47.400709    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-nmqfs"
Sep 10 13:29:47.496516 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:47.496479    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwx5j\" (UniqueName: \"kubernetes.io/projected/1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf-kube-api-access-cwx5j\") pod \"1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf\" (UID: \"1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf\") "
Sep 10 13:29:47.498613 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:47.498568    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf-kube-api-access-cwx5j" (OuterVolumeSpecName: "kube-api-access-cwx5j") pod "1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf" (UID: "1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf"). InnerVolumeSpecName "kube-api-access-cwx5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:29:47.597991 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:47.597953    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwx5j\" (UniqueName: \"kubernetes.io/projected/1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf-kube-api-access-cwx5j\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:29:48.166286 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:48.166245    2551 generic.go:356] "Generic (PLEG): container finished" podID="1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf" containerID="d812366a52d968f80fc416cd973154ae1cd6959f6917d6124e0cda677bed18b0" exitCode=0
Sep 10 13:29:48.166706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:48.166305    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-nmqfs"
Sep 10 13:29:48.166706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:48.166328    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-nmqfs" event={"ID":"1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf","Type":"ContainerDied","Data":"d812366a52d968f80fc416cd973154ae1cd6959f6917d6124e0cda677bed18b0"}
Sep 10 13:29:48.166706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:48.166372    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-nmqfs" event={"ID":"1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf","Type":"ContainerDied","Data":"b2b72e4191ac46d7660f3f795aa3cfbee5c9691312bafda1df939e356cac4bfe"}
Sep 10 13:29:48.166706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:48.166388    2551 scope.go:117] "RemoveContainer" containerID="d812366a52d968f80fc416cd973154ae1cd6959f6917d6124e0cda677bed18b0"
Sep 10 13:29:48.174328 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:48.174306    2551 scope.go:117] "RemoveContainer" containerID="d812366a52d968f80fc416cd973154ae1cd6959f6917d6124e0cda677bed18b0"
Sep 10 13:29:48.174620 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:29:48.174599    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d812366a52d968f80fc416cd973154ae1cd6959f6917d6124e0cda677bed18b0\": container with ID starting with d812366a52d968f80fc416cd973154ae1cd6959f6917d6124e0cda677bed18b0 not found: ID does not exist" containerID="d812366a52d968f80fc416cd973154ae1cd6959f6917d6124e0cda677bed18b0"
Sep 10 13:29:48.174704 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:48.174632    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d812366a52d968f80fc416cd973154ae1cd6959f6917d6124e0cda677bed18b0"} err="failed to get container status \"d812366a52d968f80fc416cd973154ae1cd6959f6917d6124e0cda677bed18b0\": rpc error: code = NotFound desc = could not find container \"d812366a52d968f80fc416cd973154ae1cd6959f6917d6124e0cda677bed18b0\": container with ID starting with d812366a52d968f80fc416cd973154ae1cd6959f6917d6124e0cda677bed18b0 not found: ID does not exist"
Sep 10 13:29:48.189953 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:48.189930    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nmqfs"]
Sep 10 13:29:48.194652 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:48.194630    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-nmqfs"]
Sep 10 13:29:49.710791 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:49.710759    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf" path="/var/lib/kubelet/pods/1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf/volumes"
Sep 10 13:29:56.157145 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:29:56.157117    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"
Sep 10 13:30:16.783743 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:16.783709    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-bb65x"]
Sep 10 13:30:16.784129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:16.784035    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf" containerName="authorino"
Sep 10 13:30:16.784129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:16.784047    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf" containerName="authorino"
Sep 10 13:30:16.784129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:16.784108    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e5cc9eb-31b6-4cb5-82a2-04b6cd82e5cf" containerName="authorino"
Sep 10 13:30:16.791083 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:16.791059    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-bb65x"
Sep 10 13:30:16.799597 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:16.799546    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-bb65x"]
Sep 10 13:30:16.931445 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:16.931408    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-bb65x"]
Sep 10 13:30:16.931725 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:30:16.931699    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-s862w], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-bb65x" podUID="86aa9489-14f5-4299-9966-e531f0f8fd45"
Sep 10 13:30:16.939929 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:16.939906    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s862w\" (UniqueName: \"kubernetes.io/projected/86aa9489-14f5-4299-9966-e531f0f8fd45-kube-api-access-s862w\") pod \"authorino-557f876f69-bb65x\" (UID: \"86aa9489-14f5-4299-9966-e531f0f8fd45\") " pod="kuadrant-system/authorino-557f876f69-bb65x"
Sep 10 13:30:17.015177 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.015153    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-kdsrw"]
Sep 10 13:30:17.018029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.018007    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-kdsrw"
Sep 10 13:30:17.021351 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.021329    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 13:30:17.041312 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.041248    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s862w\" (UniqueName: \"kubernetes.io/projected/86aa9489-14f5-4299-9966-e531f0f8fd45-kube-api-access-s862w\") pod \"authorino-557f876f69-bb65x\" (UID: \"86aa9489-14f5-4299-9966-e531f0f8fd45\") " pod="kuadrant-system/authorino-557f876f69-bb65x"
Sep 10 13:30:17.044450 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.044425    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-kdsrw"]
Sep 10 13:30:17.062167 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.062139    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s862w\" (UniqueName: \"kubernetes.io/projected/86aa9489-14f5-4299-9966-e531f0f8fd45-kube-api-access-s862w\") pod \"authorino-557f876f69-bb65x\" (UID: \"86aa9489-14f5-4299-9966-e531f0f8fd45\") " pod="kuadrant-system/authorino-557f876f69-bb65x"
Sep 10 13:30:17.109273 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.109243    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-kdsrw"]
Sep 10 13:30:17.109498 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:30:17.109479    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-njth6 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-kdsrw" podUID="58fc526c-9f2b-4fd2-9803-e2e95bd86a37"
Sep 10 13:30:17.142126 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.142096    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njth6\" (UniqueName: \"kubernetes.io/projected/58fc526c-9f2b-4fd2-9803-e2e95bd86a37-kube-api-access-njth6\") pod \"authorino-6d4dfc649f-kdsrw\" (UID: \"58fc526c-9f2b-4fd2-9803-e2e95bd86a37\") " pod="kuadrant-system/authorino-6d4dfc649f-kdsrw"
Sep 10 13:30:17.142242 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.142148    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58fc526c-9f2b-4fd2-9803-e2e95bd86a37-tls-cert\") pod \"authorino-6d4dfc649f-kdsrw\" (UID: \"58fc526c-9f2b-4fd2-9803-e2e95bd86a37\") " pod="kuadrant-system/authorino-6d4dfc649f-kdsrw"
Sep 10 13:30:17.205789 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.205759    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c5dc6d464-khzl6"]
Sep 10 13:30:17.207965 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.207949    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c5dc6d464-khzl6"
Sep 10 13:30:17.223152 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.223124    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c5dc6d464-khzl6"]
Sep 10 13:30:17.242593 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.242553    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58fc526c-9f2b-4fd2-9803-e2e95bd86a37-tls-cert\") pod \"authorino-6d4dfc649f-kdsrw\" (UID: \"58fc526c-9f2b-4fd2-9803-e2e95bd86a37\") " pod="kuadrant-system/authorino-6d4dfc649f-kdsrw"
Sep 10 13:30:17.242715 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.242654    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njth6\" (UniqueName: \"kubernetes.io/projected/58fc526c-9f2b-4fd2-9803-e2e95bd86a37-kube-api-access-njth6\") pod \"authorino-6d4dfc649f-kdsrw\" (UID: \"58fc526c-9f2b-4fd2-9803-e2e95bd86a37\") " pod="kuadrant-system/authorino-6d4dfc649f-kdsrw"
Sep 10 13:30:17.244927 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.244901    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58fc526c-9f2b-4fd2-9803-e2e95bd86a37-tls-cert\") pod \"authorino-6d4dfc649f-kdsrw\" (UID: \"58fc526c-9f2b-4fd2-9803-e2e95bd86a37\") " pod="kuadrant-system/authorino-6d4dfc649f-kdsrw"
Sep 10 13:30:17.257788 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.257768    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njth6\" (UniqueName: \"kubernetes.io/projected/58fc526c-9f2b-4fd2-9803-e2e95bd86a37-kube-api-access-njth6\") pod \"authorino-6d4dfc649f-kdsrw\" (UID: \"58fc526c-9f2b-4fd2-9803-e2e95bd86a37\") " pod="kuadrant-system/authorino-6d4dfc649f-kdsrw"
Sep 10 13:30:17.290176 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.290152    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-kdsrw"
Sep 10 13:30:17.290278 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.290175    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-bb65x"
Sep 10 13:30:17.296175 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.296122    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-kdsrw"
Sep 10 13:30:17.298946 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.298928    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-bb65x"
Sep 10 13:30:17.343160 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.343140    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c68fa916-43e3-46e2-b497-8b2d66aa7609-tls-cert\") pod \"authorino-7c5dc6d464-khzl6\" (UID: \"c68fa916-43e3-46e2-b497-8b2d66aa7609\") " pod="kuadrant-system/authorino-7c5dc6d464-khzl6"
Sep 10 13:30:17.343252 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.343231    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb5qv\" (UniqueName: \"kubernetes.io/projected/c68fa916-43e3-46e2-b497-8b2d66aa7609-kube-api-access-hb5qv\") pod \"authorino-7c5dc6d464-khzl6\" (UID: \"c68fa916-43e3-46e2-b497-8b2d66aa7609\") " pod="kuadrant-system/authorino-7c5dc6d464-khzl6"
Sep 10 13:30:17.444359 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.444339    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njth6\" (UniqueName: \"kubernetes.io/projected/58fc526c-9f2b-4fd2-9803-e2e95bd86a37-kube-api-access-njth6\") pod \"58fc526c-9f2b-4fd2-9803-e2e95bd86a37\" (UID: \"58fc526c-9f2b-4fd2-9803-e2e95bd86a37\") "
Sep 10 13:30:17.444450 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.444384    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s862w\" (UniqueName: \"kubernetes.io/projected/86aa9489-14f5-4299-9966-e531f0f8fd45-kube-api-access-s862w\") pod \"86aa9489-14f5-4299-9966-e531f0f8fd45\" (UID: \"86aa9489-14f5-4299-9966-e531f0f8fd45\") "
Sep 10 13:30:17.444582 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.444556    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58fc526c-9f2b-4fd2-9803-e2e95bd86a37-tls-cert\") pod \"58fc526c-9f2b-4fd2-9803-e2e95bd86a37\" (UID: \"58fc526c-9f2b-4fd2-9803-e2e95bd86a37\") "
Sep 10 13:30:17.444740 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.444723    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb5qv\" (UniqueName: \"kubernetes.io/projected/c68fa916-43e3-46e2-b497-8b2d66aa7609-kube-api-access-hb5qv\") pod \"authorino-7c5dc6d464-khzl6\" (UID: \"c68fa916-43e3-46e2-b497-8b2d66aa7609\") " pod="kuadrant-system/authorino-7c5dc6d464-khzl6"
Sep 10 13:30:17.444813 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.444755    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c68fa916-43e3-46e2-b497-8b2d66aa7609-tls-cert\") pod \"authorino-7c5dc6d464-khzl6\" (UID: \"c68fa916-43e3-46e2-b497-8b2d66aa7609\") " pod="kuadrant-system/authorino-7c5dc6d464-khzl6"
Sep 10 13:30:17.446425 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.446395    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58fc526c-9f2b-4fd2-9803-e2e95bd86a37-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "58fc526c-9f2b-4fd2-9803-e2e95bd86a37" (UID: "58fc526c-9f2b-4fd2-9803-e2e95bd86a37"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:30:17.446425 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.446407    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86aa9489-14f5-4299-9966-e531f0f8fd45-kube-api-access-s862w" (OuterVolumeSpecName: "kube-api-access-s862w") pod "86aa9489-14f5-4299-9966-e531f0f8fd45" (UID: "86aa9489-14f5-4299-9966-e531f0f8fd45"). InnerVolumeSpecName "kube-api-access-s862w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:30:17.446549 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.446528    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58fc526c-9f2b-4fd2-9803-e2e95bd86a37-kube-api-access-njth6" (OuterVolumeSpecName: "kube-api-access-njth6") pod "58fc526c-9f2b-4fd2-9803-e2e95bd86a37" (UID: "58fc526c-9f2b-4fd2-9803-e2e95bd86a37"). InnerVolumeSpecName "kube-api-access-njth6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:30:17.447029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.447012    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c68fa916-43e3-46e2-b497-8b2d66aa7609-tls-cert\") pod \"authorino-7c5dc6d464-khzl6\" (UID: \"c68fa916-43e3-46e2-b497-8b2d66aa7609\") " pod="kuadrant-system/authorino-7c5dc6d464-khzl6"
Sep 10 13:30:17.455429 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.455408    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb5qv\" (UniqueName: \"kubernetes.io/projected/c68fa916-43e3-46e2-b497-8b2d66aa7609-kube-api-access-hb5qv\") pod \"authorino-7c5dc6d464-khzl6\" (UID: \"c68fa916-43e3-46e2-b497-8b2d66aa7609\") " pod="kuadrant-system/authorino-7c5dc6d464-khzl6"
Sep 10 13:30:17.516907 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.516881    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c5dc6d464-khzl6"
Sep 10 13:30:17.545468 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.545439    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s862w\" (UniqueName: \"kubernetes.io/projected/86aa9489-14f5-4299-9966-e531f0f8fd45-kube-api-access-s862w\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:30:17.545468 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.545469    2551 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58fc526c-9f2b-4fd2-9803-e2e95bd86a37-tls-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:30:17.545645 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.545493    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njth6\" (UniqueName: \"kubernetes.io/projected/58fc526c-9f2b-4fd2-9803-e2e95bd86a37-kube-api-access-njth6\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:30:17.784976 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:17.784956    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c5dc6d464-khzl6"]
Sep 10 13:30:17.787045 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:30:17.787019    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc68fa916_43e3_46e2_b497_8b2d66aa7609.slice/crio-1a4d9f833dac23ca8608ad22b89169ece11a644d32e6d36f5f0d6c2a08718c65 WatchSource:0}: Error finding container 1a4d9f833dac23ca8608ad22b89169ece11a644d32e6d36f5f0d6c2a08718c65: Status 404 returned error can't find the container with id 1a4d9f833dac23ca8608ad22b89169ece11a644d32e6d36f5f0d6c2a08718c65
Sep 10 13:30:18.295448 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:18.295422    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-bb65x"
Sep 10 13:30:18.295655 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:18.295463    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c5dc6d464-khzl6" event={"ID":"c68fa916-43e3-46e2-b497-8b2d66aa7609","Type":"ContainerStarted","Data":"1a4d9f833dac23ca8608ad22b89169ece11a644d32e6d36f5f0d6c2a08718c65"}
Sep 10 13:30:18.295655 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:18.295605    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-kdsrw"
Sep 10 13:30:18.326284 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:18.326258    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-bb65x"]
Sep 10 13:30:18.330890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:18.330867    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-bb65x"]
Sep 10 13:30:18.363284 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:18.363259    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-kdsrw"]
Sep 10 13:30:18.368469 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:18.368449    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-kdsrw"]
Sep 10 13:30:19.301186 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:19.301152    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c5dc6d464-khzl6" event={"ID":"c68fa916-43e3-46e2-b497-8b2d66aa7609","Type":"ContainerStarted","Data":"0f0b9022423534b7c777606baaec52cc3dff00b810c4877016165fa2d7204564"}
Sep 10 13:30:19.326356 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:19.326308    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7c5dc6d464-khzl6" podStartSLOduration=1.605764234 podStartE2EDuration="2.326294894s" totalImagesPullingTime="720.53066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:30:19.32439662 +0000 UTC m=+730.143435020" watchObservedRunningTime="2026-09-10 13:30:19.326294894 +0000 UTC m=+730.145333294"
Sep 10 13:30:19.371521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:19.366609    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-5wm8t"]
Sep 10 13:30:19.371521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:19.366838    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-5wm8t" podUID="e884f0af-4f1f-43ca-abc3-95964e23c6a6" containerName="authorino" containerID="cri-o://34ae02dfefb80b459f95a3b4caf4c1321dadf670bafbe9cbc458a85a2e7bd036" gracePeriod=30
Sep 10 13:30:19.614062 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:19.614035    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-5wm8t"
Sep 10 13:30:19.709248 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:19.709216    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58fc526c-9f2b-4fd2-9803-e2e95bd86a37" path="/var/lib/kubelet/pods/58fc526c-9f2b-4fd2-9803-e2e95bd86a37/volumes"
Sep 10 13:30:19.709447 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:19.709435    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86aa9489-14f5-4299-9966-e531f0f8fd45" path="/var/lib/kubelet/pods/86aa9489-14f5-4299-9966-e531f0f8fd45/volumes"
Sep 10 13:30:19.763468 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:19.763441    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kh5c\" (UniqueName: \"kubernetes.io/projected/e884f0af-4f1f-43ca-abc3-95964e23c6a6-kube-api-access-6kh5c\") pod \"e884f0af-4f1f-43ca-abc3-95964e23c6a6\" (UID: \"e884f0af-4f1f-43ca-abc3-95964e23c6a6\") "
Sep 10 13:30:19.765377 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:19.765348    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e884f0af-4f1f-43ca-abc3-95964e23c6a6-kube-api-access-6kh5c" (OuterVolumeSpecName: "kube-api-access-6kh5c") pod "e884f0af-4f1f-43ca-abc3-95964e23c6a6" (UID: "e884f0af-4f1f-43ca-abc3-95964e23c6a6"). InnerVolumeSpecName "kube-api-access-6kh5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:30:19.864786 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:19.864723    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kh5c\" (UniqueName: \"kubernetes.io/projected/e884f0af-4f1f-43ca-abc3-95964e23c6a6-kube-api-access-6kh5c\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:30:20.307056 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:20.307019    2551 generic.go:356] "Generic (PLEG): container finished" podID="e884f0af-4f1f-43ca-abc3-95964e23c6a6" containerID="34ae02dfefb80b459f95a3b4caf4c1321dadf670bafbe9cbc458a85a2e7bd036" exitCode=0
Sep 10 13:30:20.307452 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:20.307063    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-5wm8t"
Sep 10 13:30:20.307452 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:20.307107    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-5wm8t" event={"ID":"e884f0af-4f1f-43ca-abc3-95964e23c6a6","Type":"ContainerDied","Data":"34ae02dfefb80b459f95a3b4caf4c1321dadf670bafbe9cbc458a85a2e7bd036"}
Sep 10 13:30:20.307452 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:20.307155    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-5wm8t" event={"ID":"e884f0af-4f1f-43ca-abc3-95964e23c6a6","Type":"ContainerDied","Data":"fd18b9049a5c9d5e428b529304982a1f09d06f202da92b243705442511c60850"}
Sep 10 13:30:20.307452 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:20.307179    2551 scope.go:117] "RemoveContainer" containerID="34ae02dfefb80b459f95a3b4caf4c1321dadf670bafbe9cbc458a85a2e7bd036"
Sep 10 13:30:20.315374 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:20.315359    2551 scope.go:117] "RemoveContainer" containerID="34ae02dfefb80b459f95a3b4caf4c1321dadf670bafbe9cbc458a85a2e7bd036"
Sep 10 13:30:20.315647 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:30:20.315631    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34ae02dfefb80b459f95a3b4caf4c1321dadf670bafbe9cbc458a85a2e7bd036\": container with ID starting with 34ae02dfefb80b459f95a3b4caf4c1321dadf670bafbe9cbc458a85a2e7bd036 not found: ID does not exist" containerID="34ae02dfefb80b459f95a3b4caf4c1321dadf670bafbe9cbc458a85a2e7bd036"
Sep 10 13:30:20.315699 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:20.315654    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34ae02dfefb80b459f95a3b4caf4c1321dadf670bafbe9cbc458a85a2e7bd036"} err="failed to get container status \"34ae02dfefb80b459f95a3b4caf4c1321dadf670bafbe9cbc458a85a2e7bd036\": rpc error: code = NotFound desc = could not find container \"34ae02dfefb80b459f95a3b4caf4c1321dadf670bafbe9cbc458a85a2e7bd036\": container with ID starting with 34ae02dfefb80b459f95a3b4caf4c1321dadf670bafbe9cbc458a85a2e7bd036 not found: ID does not exist"
Sep 10 13:30:20.337210 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:20.337185    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-5wm8t"]
Sep 10 13:30:20.341202 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:20.341177    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-5wm8t"]
Sep 10 13:30:21.709737 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:30:21.709703    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e884f0af-4f1f-43ca-abc3-95964e23c6a6" path="/var/lib/kubelet/pods/e884f0af-4f1f-43ca-abc3-95964e23c6a6/volumes"
Sep 10 13:31:03.285478 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:03.285445    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:31:11.158727 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:11.158689    2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jhkw6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 13:31:11.159107 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:11.158758    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6" podUID="36051b6e-ece1-419c-9a57-3610df345db8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 13:31:13.544820 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:13.544785    2551 generic.go:356] "Generic (PLEG): container finished" podID="36051b6e-ece1-419c-9a57-3610df345db8" containerID="15958851f2c2395ac1bed91b3f8d90fa846ee103b9e09ffe541304af7c17f0c4" exitCode=0
Sep 10 13:31:13.545224 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:13.544860    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6" event={"ID":"36051b6e-ece1-419c-9a57-3610df345db8","Type":"ContainerDied","Data":"15958851f2c2395ac1bed91b3f8d90fa846ee103b9e09ffe541304af7c17f0c4"}
Sep 10 13:31:13.545272 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:13.545228    2551 scope.go:117] "RemoveContainer" containerID="15958851f2c2395ac1bed91b3f8d90fa846ee103b9e09ffe541304af7c17f0c4"
Sep 10 13:31:13.546196 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:13.546167    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:31:13.546249 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:13.546236    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:31:13.546284 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:13.546270    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:31:14.553830 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:14.553785    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6" event={"ID":"36051b6e-ece1-419c-9a57-3610df345db8","Type":"ContainerStarted","Data":"ce10f373b8e0f644caf664519c66ce06ebed57e4c58f51957c7aa42f75971778"}
Sep 10 13:31:14.673405 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:14.673368    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:31:14.673661 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:14.673638    2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jhkw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Sep 10 13:31:14.673711 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:14.673689    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6" podUID="36051b6e-ece1-419c-9a57-3610df345db8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Sep 10 13:31:15.673661 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:15.673625    2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jhkw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Sep 10 13:31:15.674038 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:15.673675    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6" podUID="36051b6e-ece1-419c-9a57-3610df345db8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Sep 10 13:31:16.673587 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:16.673537    2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jhkw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Sep 10 13:31:16.673756 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:16.673612    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6" podUID="36051b6e-ece1-419c-9a57-3610df345db8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Sep 10 13:31:17.673074 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:17.673038    2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jhkw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Sep 10 13:31:17.673257 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:17.673091    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6" podUID="36051b6e-ece1-419c-9a57-3610df345db8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Sep 10 13:31:18.673924 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:18.673883    2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jhkw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Sep 10 13:31:18.674319 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:18.673953    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6" podUID="36051b6e-ece1-419c-9a57-3610df345db8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Sep 10 13:31:19.676730 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:19.676701    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:31:19.677151 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:19.676945    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:31:19.677532 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:19.677514    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jhkw6"
Sep 10 13:31:25.345707 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:25.345669    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:31:30.018983 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:30.018945    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:31:41.489544 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.489463    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"]
Sep 10 13:31:41.489916 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.489813    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e884f0af-4f1f-43ca-abc3-95964e23c6a6" containerName="authorino"
Sep 10 13:31:41.489916 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.489825    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e884f0af-4f1f-43ca-abc3-95964e23c6a6" containerName="authorino"
Sep 10 13:31:41.489916 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.489902    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e884f0af-4f1f-43ca-abc3-95964e23c6a6" containerName="authorino"
Sep 10 13:31:41.491940 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.491924    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.498810 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.498783    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 10 13:31:41.498945 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.498790    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nhrvj\""
Sep 10 13:31:41.498945 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.498840    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 13:31:41.508962 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.508936    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 13:31:41.550835 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.550804    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f633bbf-c464-4379-967e-9e4b68d2c8ba-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.550962 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.550854    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f633bbf-c464-4379-967e-9e4b68d2c8ba-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.550962 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.550900    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f633bbf-c464-4379-967e-9e4b68d2c8ba-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.551037 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.550974    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f6wn\" (UniqueName: \"kubernetes.io/projected/6f633bbf-c464-4379-967e-9e4b68d2c8ba-kube-api-access-2f6wn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.551037 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.551002    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f633bbf-c464-4379-967e-9e4b68d2c8ba-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.551037 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.551021    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f633bbf-c464-4379-967e-9e4b68d2c8ba-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.566944 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.566917    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"]
Sep 10 13:31:41.652503 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.652470    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f633bbf-c464-4379-967e-9e4b68d2c8ba-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.652679 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.652520    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f633bbf-c464-4379-967e-9e4b68d2c8ba-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.652679 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.652552    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f633bbf-c464-4379-967e-9e4b68d2c8ba-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.652679 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.652608    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f6wn\" (UniqueName: \"kubernetes.io/projected/6f633bbf-c464-4379-967e-9e4b68d2c8ba-kube-api-access-2f6wn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.652679 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.652631    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f633bbf-c464-4379-967e-9e4b68d2c8ba-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.652856 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.652749    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f633bbf-c464-4379-967e-9e4b68d2c8ba-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.652999 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.652977    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f633bbf-c464-4379-967e-9e4b68d2c8ba-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.653070 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.653009    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f633bbf-c464-4379-967e-9e4b68d2c8ba-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.653134 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.653106    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f633bbf-c464-4379-967e-9e4b68d2c8ba-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.654683 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.654665    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f633bbf-c464-4379-967e-9e4b68d2c8ba-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.655026 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.655008    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f633bbf-c464-4379-967e-9e4b68d2c8ba-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.668721 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.668696    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f6wn\" (UniqueName: \"kubernetes.io/projected/6f633bbf-c464-4379-967e-9e4b68d2c8ba-kube-api-access-2f6wn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf\" (UID: \"6f633bbf-c464-4379-967e-9e4b68d2c8ba\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:41.801864 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:41.801789    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:42.078110 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:42.078034    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"]
Sep 10 13:31:42.080896 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:31:42.080872    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f633bbf_c464_4379_967e_9e4b68d2c8ba.slice/crio-a81c21965053e8605eac2502ddd739b5adf2ed5d3da6c33dbe990fa992ac2cb5 WatchSource:0}: Error finding container a81c21965053e8605eac2502ddd739b5adf2ed5d3da6c33dbe990fa992ac2cb5: Status 404 returned error can't find the container with id a81c21965053e8605eac2502ddd739b5adf2ed5d3da6c33dbe990fa992ac2cb5
Sep 10 13:31:42.513432 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:42.513396    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:31:42.686380 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:42.686334    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf" event={"ID":"6f633bbf-c464-4379-967e-9e4b68d2c8ba","Type":"ContainerStarted","Data":"a81c21965053e8605eac2502ddd739b5adf2ed5d3da6c33dbe990fa992ac2cb5"}
Sep 10 13:31:43.692609 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:43.692558    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf" event={"ID":"6f633bbf-c464-4379-967e-9e4b68d2c8ba","Type":"ContainerStarted","Data":"da6f5ec8a6a4541fc8ac52f93dee28c711f57ae94bd0ecfb9c43c7a97a45a997"}
Sep 10 13:31:43.734275 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:43.734227    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf" podStartSLOduration=1.698647613 podStartE2EDuration="2.734212378s" totalImagesPullingTime="1.035564765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:43.733430731 +0000 UTC m=+814.552469132" watchObservedRunningTime="2026-09-10 13:31:43.734212378 +0000 UTC m=+814.553250772"
Sep 10 13:31:51.802665 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:51.802632    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:51.803062 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:51.802701    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:51.815385 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:51.815359    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:52.741556 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:52.741525    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf"
Sep 10 13:31:55.030932 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:31:55.030895    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:32:07.900814 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:07.900784    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"]
Sep 10 13:32:07.903049 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:07.903033    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:07.906869 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:07.906847    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 10 13:32:07.944870 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:07.944840    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"]
Sep 10 13:32:07.977784 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:07.977760    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:07.977890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:07.977808    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:07.977890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:07.977854    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng8v4\" (UniqueName: \"kubernetes.io/projected/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-kube-api-access-ng8v4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:07.977890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:07.977880    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:07.977994 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:07.977894    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:07.977994 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:07.977920    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:08.079155 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.079118    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng8v4\" (UniqueName: \"kubernetes.io/projected/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-kube-api-access-ng8v4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:08.079155 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.079159    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:08.079347 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.079176    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:08.079347 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.079196    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:08.079347 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.079265    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:08.079347 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.079319    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:08.079640 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.079616    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:08.079700 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.079671    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:08.079736 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.079719    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:08.081428 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.081407    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:08.081704 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.081688    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:08.089751 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.089728    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng8v4\" (UniqueName: \"kubernetes.io/projected/dd1db447-4f7a-42fb-bb74-d4a6610bc09b-kube-api-access-ng8v4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm\" (UID: \"dd1db447-4f7a-42fb-bb74-d4a6610bc09b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:08.212371 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.212288    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:08.511794 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.511768    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"]
Sep 10 13:32:08.512971 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:32:08.512947    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd1db447_4f7a_42fb_bb74_d4a6610bc09b.slice/crio-54545dcef395110e7486ff26c01095bdb073c4a769f2820518c8e93453336247 WatchSource:0}: Error finding container 54545dcef395110e7486ff26c01095bdb073c4a769f2820518c8e93453336247: Status 404 returned error can't find the container with id 54545dcef395110e7486ff26c01095bdb073c4a769f2820518c8e93453336247
Sep 10 13:32:08.806471 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.806397    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm" event={"ID":"dd1db447-4f7a-42fb-bb74-d4a6610bc09b","Type":"ContainerStarted","Data":"9f883cea1cc9790551158b1b6e206d62debf8f03dedcc55febb14f95665a01b6"}
Sep 10 13:32:08.806471 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.806449    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm" event={"ID":"dd1db447-4f7a-42fb-bb74-d4a6610bc09b","Type":"ContainerStarted","Data":"54545dcef395110e7486ff26c01095bdb073c4a769f2820518c8e93453336247"}
Sep 10 13:32:08.864107 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:08.864058    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm" podStartSLOduration=1.687085627 podStartE2EDuration="1.864040507s" totalImagesPullingTime="176.95488ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:32:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:32:08.862470022 +0000 UTC m=+839.681508421" watchObservedRunningTime="2026-09-10 13:32:08.864040507 +0000 UTC m=+839.683078908"
Sep 10 13:32:09.733950 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:09.733920    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:32:16.775189 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:16.775147    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:32:18.213332 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:18.213287    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:18.213773 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:18.213347    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:18.225580 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:18.225557    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:18.862288 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:18.862259    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm"
Sep 10 13:32:20.887009 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:20.886980    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"]
Sep 10 13:32:20.890118 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:20.890101    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:20.893095 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:20.893070    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 13:32:20.906179 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:20.906157    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"]
Sep 10 13:32:20.993911 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:20.993888    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:20.994021 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:20.993944    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:20.994064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:20.994017    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:20.994064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:20.994041    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:20.994131 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:20.994064    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:20.994131 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:20.994085    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4j78\" (UniqueName: \"kubernetes.io/projected/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-kube-api-access-w4j78\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:21.095115 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.095089    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:21.095115 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.095119    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:21.095273 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.095137    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:21.095273 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.095160    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4j78\" (UniqueName: \"kubernetes.io/projected/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-kube-api-access-w4j78\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:21.095273 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.095213    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:21.095397 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.095270    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:21.095513 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.095492    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:21.095643 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.095614    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:21.095761 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.095661    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:21.097406 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.097386    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:21.097550 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.097531    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:21.106003 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.105981    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4j78\" (UniqueName: \"kubernetes.io/projected/98f4f9fd-0d9f-42d9-90f9-0148df14a83b-kube-api-access-w4j78\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp\" (UID: \"98f4f9fd-0d9f-42d9-90f9-0148df14a83b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:21.198942 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.198882    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:21.474434 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.474406    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"]
Sep 10 13:32:21.477655 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:32:21.477629    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98f4f9fd_0d9f_42d9_90f9_0148df14a83b.slice/crio-9876d31ec5c14c3da6414c03bf8dd0affe85b6b214566ba091efcf0698593779 WatchSource:0}: Error finding container 9876d31ec5c14c3da6414c03bf8dd0affe85b6b214566ba091efcf0698593779: Status 404 returned error can't find the container with id 9876d31ec5c14c3da6414c03bf8dd0affe85b6b214566ba091efcf0698593779
Sep 10 13:32:21.865892 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.865803    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp" event={"ID":"98f4f9fd-0d9f-42d9-90f9-0148df14a83b","Type":"ContainerStarted","Data":"e77dafd7107129b807cf2dd7f179ba4a9d05c6ba484ee53e8238d444d8df3977"}
Sep 10 13:32:21.865892 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.865849    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp" event={"ID":"98f4f9fd-0d9f-42d9-90f9-0148df14a83b","Type":"ContainerStarted","Data":"9876d31ec5c14c3da6414c03bf8dd0affe85b6b214566ba091efcf0698593779"}
Sep 10 13:32:21.889839 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:21.889786    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp" podStartSLOduration=1.700864489 podStartE2EDuration="1.889769147s" totalImagesPullingTime="188.904658ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:32:21.888403256 +0000 UTC m=+852.707441655" watchObservedRunningTime="2026-09-10 13:32:21.889769147 +0000 UTC m=+852.708807548"
Sep 10 13:32:22.908659 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:22.908617    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:32:31.199753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:31.199720    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:31.200176 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:31.199802    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:31.212156 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:31.212131    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:32:31.924899 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:32:31.924869    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp"
Sep 10 13:33:09.646733 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:33:09.646660    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:33:09.647178 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:33:09.647137    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:34:09.711727 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:09.711696    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8c58dd797-lstql"]
Sep 10 13:34:09.714530 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:09.714505    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8c58dd797-lstql"
Sep 10 13:34:09.756289 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:09.756264    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8c58dd797-lstql"]
Sep 10 13:34:09.806337 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:09.806310    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6-tls-cert\") pod \"authorino-8c58dd797-lstql\" (UID: \"f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6\") " pod="kuadrant-system/authorino-8c58dd797-lstql"
Sep 10 13:34:09.806470 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:09.806356    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vd2m\" (UniqueName: \"kubernetes.io/projected/f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6-kube-api-access-7vd2m\") pod \"authorino-8c58dd797-lstql\" (UID: \"f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6\") " pod="kuadrant-system/authorino-8c58dd797-lstql"
Sep 10 13:34:09.907503 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:09.907472    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6-tls-cert\") pod \"authorino-8c58dd797-lstql\" (UID: \"f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6\") " pod="kuadrant-system/authorino-8c58dd797-lstql"
Sep 10 13:34:09.907677 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:09.907513    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vd2m\" (UniqueName: \"kubernetes.io/projected/f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6-kube-api-access-7vd2m\") pod \"authorino-8c58dd797-lstql\" (UID: \"f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6\") " pod="kuadrant-system/authorino-8c58dd797-lstql"
Sep 10 13:34:09.909834 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:09.909812    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6-tls-cert\") pod \"authorino-8c58dd797-lstql\" (UID: \"f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6\") " pod="kuadrant-system/authorino-8c58dd797-lstql"
Sep 10 13:34:09.921957 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:09.921932    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vd2m\" (UniqueName: \"kubernetes.io/projected/f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6-kube-api-access-7vd2m\") pod \"authorino-8c58dd797-lstql\" (UID: \"f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6\") " pod="kuadrant-system/authorino-8c58dd797-lstql"
Sep 10 13:34:10.023203 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:10.023159    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8c58dd797-lstql"
Sep 10 13:34:10.300648 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:10.300558    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8c58dd797-lstql"]
Sep 10 13:34:10.303780 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:34:10.303752    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4e2cf2d_42d5_4a9b_82a6_d3dbf13623b6.slice/crio-e409f7a6b70d54a007d4c0403e7fb616ad7a67b43be976a1b3c83130fc5e7898 WatchSource:0}: Error finding container e409f7a6b70d54a007d4c0403e7fb616ad7a67b43be976a1b3c83130fc5e7898: Status 404 returned error can't find the container with id e409f7a6b70d54a007d4c0403e7fb616ad7a67b43be976a1b3c83130fc5e7898
Sep 10 13:34:10.358863 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:10.358834    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8c58dd797-lstql" event={"ID":"f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6","Type":"ContainerStarted","Data":"e409f7a6b70d54a007d4c0403e7fb616ad7a67b43be976a1b3c83130fc5e7898"}
Sep 10 13:34:13.380594 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:13.380523    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8c58dd797-lstql" event={"ID":"f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6","Type":"ContainerStarted","Data":"c6779a421a4f768d296a7cf7aedce96c9cd5b27830b550aadb9151fd791f8a13"}
Sep 10 13:34:13.413828 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:13.413759    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8c58dd797-lstql" podStartSLOduration=2.2718412089999998 podStartE2EDuration="4.413735337s" totalImagesPullingTime="2.141894128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:34:13.410291867 +0000 UTC m=+964.229330269" watchObservedRunningTime="2026-09-10 13:34:13.413735337 +0000 UTC m=+964.232773738"
Sep 10 13:34:13.492560 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:13.492523    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c5dc6d464-khzl6"]
Sep 10 13:34:13.493005 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:13.492974    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7c5dc6d464-khzl6" podUID="c68fa916-43e3-46e2-b497-8b2d66aa7609" containerName="authorino" containerID="cri-o://0f0b9022423534b7c777606baaec52cc3dff00b810c4877016165fa2d7204564" gracePeriod=30
Sep 10 13:34:13.739890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:13.739866    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c5dc6d464-khzl6"
Sep 10 13:34:13.841187 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:13.841150    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hb5qv\" (UniqueName: \"kubernetes.io/projected/c68fa916-43e3-46e2-b497-8b2d66aa7609-kube-api-access-hb5qv\") pod \"c68fa916-43e3-46e2-b497-8b2d66aa7609\" (UID: \"c68fa916-43e3-46e2-b497-8b2d66aa7609\") "
Sep 10 13:34:13.841361 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:13.841249    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c68fa916-43e3-46e2-b497-8b2d66aa7609-tls-cert\") pod \"c68fa916-43e3-46e2-b497-8b2d66aa7609\" (UID: \"c68fa916-43e3-46e2-b497-8b2d66aa7609\") "
Sep 10 13:34:13.843304 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:13.843269    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c68fa916-43e3-46e2-b497-8b2d66aa7609-kube-api-access-hb5qv" (OuterVolumeSpecName: "kube-api-access-hb5qv") pod "c68fa916-43e3-46e2-b497-8b2d66aa7609" (UID: "c68fa916-43e3-46e2-b497-8b2d66aa7609"). InnerVolumeSpecName "kube-api-access-hb5qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:34:13.853436 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:13.853409    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c68fa916-43e3-46e2-b497-8b2d66aa7609-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c68fa916-43e3-46e2-b497-8b2d66aa7609" (UID: "c68fa916-43e3-46e2-b497-8b2d66aa7609"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:34:13.942218 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:13.942119    2551 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c68fa916-43e3-46e2-b497-8b2d66aa7609-tls-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:34:13.942218 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:13.942153    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb5qv\" (UniqueName: \"kubernetes.io/projected/c68fa916-43e3-46e2-b497-8b2d66aa7609-kube-api-access-hb5qv\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:34:14.387360 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:14.387324    2551 generic.go:356] "Generic (PLEG): container finished" podID="c68fa916-43e3-46e2-b497-8b2d66aa7609" containerID="0f0b9022423534b7c777606baaec52cc3dff00b810c4877016165fa2d7204564" exitCode=0
Sep 10 13:34:14.387794 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:14.387384    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c5dc6d464-khzl6"
Sep 10 13:34:14.387794 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:14.387405    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c5dc6d464-khzl6" event={"ID":"c68fa916-43e3-46e2-b497-8b2d66aa7609","Type":"ContainerDied","Data":"0f0b9022423534b7c777606baaec52cc3dff00b810c4877016165fa2d7204564"}
Sep 10 13:34:14.387794 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:14.387447    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c5dc6d464-khzl6" event={"ID":"c68fa916-43e3-46e2-b497-8b2d66aa7609","Type":"ContainerDied","Data":"1a4d9f833dac23ca8608ad22b89169ece11a644d32e6d36f5f0d6c2a08718c65"}
Sep 10 13:34:14.387794 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:14.387463    2551 scope.go:117] "RemoveContainer" containerID="0f0b9022423534b7c777606baaec52cc3dff00b810c4877016165fa2d7204564"
Sep 10 13:34:14.395997 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:14.395977    2551 scope.go:117] "RemoveContainer" containerID="0f0b9022423534b7c777606baaec52cc3dff00b810c4877016165fa2d7204564"
Sep 10 13:34:14.396302 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:34:14.396284    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f0b9022423534b7c777606baaec52cc3dff00b810c4877016165fa2d7204564\": container with ID starting with 0f0b9022423534b7c777606baaec52cc3dff00b810c4877016165fa2d7204564 not found: ID does not exist" containerID="0f0b9022423534b7c777606baaec52cc3dff00b810c4877016165fa2d7204564"
Sep 10 13:34:14.396357 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:14.396311    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f0b9022423534b7c777606baaec52cc3dff00b810c4877016165fa2d7204564"} err="failed to get container status \"0f0b9022423534b7c777606baaec52cc3dff00b810c4877016165fa2d7204564\": rpc error: code = NotFound desc = could not find container \"0f0b9022423534b7c777606baaec52cc3dff00b810c4877016165fa2d7204564\": container with ID starting with 0f0b9022423534b7c777606baaec52cc3dff00b810c4877016165fa2d7204564 not found: ID does not exist"
Sep 10 13:34:14.428014 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:14.427976    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c5dc6d464-khzl6"]
Sep 10 13:34:14.433828 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:14.433799    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7c5dc6d464-khzl6"]
Sep 10 13:34:15.709729 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:15.709697    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c68fa916-43e3-46e2-b497-8b2d66aa7609" path="/var/lib/kubelet/pods/c68fa916-43e3-46e2-b497-8b2d66aa7609/volumes"
Sep 10 13:34:33.554665 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:33.554591    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7796-4765f4"]
Sep 10 13:34:33.555119 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:33.554958    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c68fa916-43e3-46e2-b497-8b2d66aa7609" containerName="authorino"
Sep 10 13:34:33.555119 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:33.554971    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c68fa916-43e3-46e2-b497-8b2d66aa7609" containerName="authorino"
Sep 10 13:34:33.555119 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:33.555042    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c68fa916-43e3-46e2-b497-8b2d66aa7609" containerName="authorino"
Sep 10 13:34:33.562973 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:33.562919    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-4765f4"
Sep 10 13:34:33.564144 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:33.564113    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7796-4765f4"]
Sep 10 13:34:33.567537 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:33.567516    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lpqwx\""
Sep 10 13:34:33.710482 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:33.710448    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn4qw\" (UniqueName: \"kubernetes.io/projected/f7855202-35fd-4989-92b5-5026729437f0-kube-api-access-zn4qw\") pod \"test-curl-7796-4765f4\" (UID: \"f7855202-35fd-4989-92b5-5026729437f0\") " pod="opendatahub/test-curl-7796-4765f4"
Sep 10 13:34:33.811135 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:33.811036    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn4qw\" (UniqueName: \"kubernetes.io/projected/f7855202-35fd-4989-92b5-5026729437f0-kube-api-access-zn4qw\") pod \"test-curl-7796-4765f4\" (UID: \"f7855202-35fd-4989-92b5-5026729437f0\") " pod="opendatahub/test-curl-7796-4765f4"
Sep 10 13:34:33.822157 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:33.822129    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn4qw\" (UniqueName: \"kubernetes.io/projected/f7855202-35fd-4989-92b5-5026729437f0-kube-api-access-zn4qw\") pod \"test-curl-7796-4765f4\" (UID: \"f7855202-35fd-4989-92b5-5026729437f0\") " pod="opendatahub/test-curl-7796-4765f4"
Sep 10 13:34:33.872922 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:33.872896    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-4765f4"
Sep 10 13:34:34.147189 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:34.147160    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7796-4765f4"]
Sep 10 13:34:34.149552 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:34:34.149526    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7855202_35fd_4989_92b5_5026729437f0.slice/crio-4b46c0e142ff96390ec3dedad610ec735ade7419462e03da6d9672f2074e52d9 WatchSource:0}: Error finding container 4b46c0e142ff96390ec3dedad610ec735ade7419462e03da6d9672f2074e52d9: Status 404 returned error can't find the container with id 4b46c0e142ff96390ec3dedad610ec735ade7419462e03da6d9672f2074e52d9
Sep 10 13:34:34.478505 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:34.478470    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7796-4765f4" event={"ID":"f7855202-35fd-4989-92b5-5026729437f0","Type":"ContainerStarted","Data":"4b46c0e142ff96390ec3dedad610ec735ade7419462e03da6d9672f2074e52d9"}
Sep 10 13:34:35.500520 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:35.500479    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:34:36.510238 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:36.510149    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7796-4765f4" event={"ID":"f7855202-35fd-4989-92b5-5026729437f0","Type":"ContainerStarted","Data":"c2bd1a76a00d47a276fab68eef772eb77c135ac8f1176d4c34d031f9d6ed5cbb"}
Sep 10 13:34:36.545445 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:36.545395    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7796-4765f4" podStartSLOduration=1.456015121 podStartE2EDuration="3.545378916s" totalImagesPullingTime="2.089363795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:34:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:34:36.544670582 +0000 UTC m=+987.363708982" watchObservedRunningTime="2026-09-10 13:34:36.545378916 +0000 UTC m=+987.364417316"
Sep 10 13:34:37.337424 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:37.337390    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7796-4765f4"]
Sep 10 13:34:37.340606 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:37.340517    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7796-4765f4"]
Sep 10 13:34:37.473429 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:37.473397    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7796-0c72b5"]
Sep 10 13:34:37.473764 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:37.473750    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7855202-35fd-4989-92b5-5026729437f0" containerName="test-curl-7796-4765f4"
Sep 10 13:34:37.473813 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:37.473766    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7855202-35fd-4989-92b5-5026729437f0" containerName="test-curl-7796-4765f4"
Sep 10 13:34:37.473849 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:37.473829    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7855202-35fd-4989-92b5-5026729437f0" containerName="test-curl-7796-4765f4"
Sep 10 13:34:37.475748 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:37.475733    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-0c72b5"
Sep 10 13:34:37.489100 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:37.489076    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7796-0c72b5"]
Sep 10 13:34:37.514483 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:34:37.514454    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7796-4765f4" container="test-curl-7796-4765f4"
Sep 10 13:34:37.544830 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:37.544797    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz54b\" (UniqueName: \"kubernetes.io/projected/3ba654ab-f4fa-4892-839b-b3b67e1bcdc4-kube-api-access-jz54b\") pod \"test-curl-7796-0c72b5\" (UID: \"3ba654ab-f4fa-4892-839b-b3b67e1bcdc4\") " pod="opendatahub/test-curl-7796-0c72b5"
Sep 10 13:34:37.645512 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:37.645433    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz54b\" (UniqueName: \"kubernetes.io/projected/3ba654ab-f4fa-4892-839b-b3b67e1bcdc4-kube-api-access-jz54b\") pod \"test-curl-7796-0c72b5\" (UID: \"3ba654ab-f4fa-4892-839b-b3b67e1bcdc4\") " pod="opendatahub/test-curl-7796-0c72b5"
Sep 10 13:34:37.656132 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:37.656094    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz54b\" (UniqueName: \"kubernetes.io/projected/3ba654ab-f4fa-4892-839b-b3b67e1bcdc4-kube-api-access-jz54b\") pod \"test-curl-7796-0c72b5\" (UID: \"3ba654ab-f4fa-4892-839b-b3b67e1bcdc4\") " pod="opendatahub/test-curl-7796-0c72b5"
Sep 10 13:34:37.784767 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:37.784736    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-0c72b5"
Sep 10 13:34:38.052617 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:38.052594    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7796-0c72b5"]
Sep 10 13:34:38.054779 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:34:38.054748    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ba654ab_f4fa_4892_839b_b3b67e1bcdc4.slice/crio-97502b4aaf1451fa29c368e451794aa48be62a6a70db687b353d6a585d05350c WatchSource:0}: Error finding container 97502b4aaf1451fa29c368e451794aa48be62a6a70db687b353d6a585d05350c: Status 404 returned error can't find the container with id 97502b4aaf1451fa29c368e451794aa48be62a6a70db687b353d6a585d05350c
Sep 10 13:34:38.520117 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:38.520075    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7796-0c72b5" event={"ID":"3ba654ab-f4fa-4892-839b-b3b67e1bcdc4","Type":"ContainerStarted","Data":"2e857422a4eb90ebd052a5e77e3b0b7b99c2642ce493da1a400f50f6555cafd4"}
Sep 10 13:34:38.520541 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:38.520124    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7796-0c72b5" event={"ID":"3ba654ab-f4fa-4892-839b-b3b67e1bcdc4","Type":"ContainerStarted","Data":"97502b4aaf1451fa29c368e451794aa48be62a6a70db687b353d6a585d05350c"}
Sep 10 13:34:38.520541 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:38.520159    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7796-4765f4" podUID="f7855202-35fd-4989-92b5-5026729437f0" containerName="test-curl-7796-4765f4" containerID="cri-o://c2bd1a76a00d47a276fab68eef772eb77c135ac8f1176d4c34d031f9d6ed5cbb" gracePeriod=2
Sep 10 13:34:38.540455 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:38.540410    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7796-0c72b5" podStartSLOduration=1.540396331 podStartE2EDuration="1.540396331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:34:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:34:38.53881518 +0000 UTC m=+989.357853579" watchObservedRunningTime="2026-09-10 13:34:38.540396331 +0000 UTC m=+989.359434731"
Sep 10 13:34:39.197328 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:39.197293    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7796-0c72b5"]
Sep 10 13:34:39.202996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:39.202972    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7796-0c72b5"]
Sep 10 13:34:39.525680 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:34:39.525650    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7796-0c72b5" container="test-curl-7796-0c72b5"
Sep 10 13:34:40.151563 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.151522    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"]
Sep 10 13:34:40.151915 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.151901    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ba654ab-f4fa-4892-839b-b3b67e1bcdc4" containerName="test-curl-7796-0c72b5"
Sep 10 13:34:40.151969 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.151917    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ba654ab-f4fa-4892-839b-b3b67e1bcdc4" containerName="test-curl-7796-0c72b5"
Sep 10 13:34:40.152010 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.151984    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ba654ab-f4fa-4892-839b-b3b67e1bcdc4" containerName="test-curl-7796-0c72b5"
Sep 10 13:34:40.153946 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.153931    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.158416 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.158395    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-deba50-openshift-default-dockercfg-8tjsv\""
Sep 10 13:34:40.186251 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.186213    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"]
Sep 10 13:34:40.271325 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.271293    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-workload-socket\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.271325 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.271327    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-credential-socket\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.271528 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.271354    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-envoy\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.271528 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.271375    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t86s6\" (UniqueName: \"kubernetes.io/projected/e232466f-ae92-4709-8a3b-8b1135ef8a9b-kube-api-access-t86s6\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.271528 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.271435    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-data\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.271528 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.271478    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-podinfo\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.271528 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.271497    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istiod-ca-cert\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.271722 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.271602    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-workload-certs\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.271722 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.271631    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-token\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.372944 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.372909    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-workload-certs\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.373122 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.372953    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-token\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.373122 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.373002    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-workload-socket\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.373122 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.373025    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-credential-socket\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.373122 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.373060    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-envoy\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.373322 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.373192    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t86s6\" (UniqueName: \"kubernetes.io/projected/e232466f-ae92-4709-8a3b-8b1135ef8a9b-kube-api-access-t86s6\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.373322 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.373258    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-data\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.373322 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.373315    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-podinfo\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.373469 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.373327    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-workload-certs\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.373469 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.373342    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istiod-ca-cert\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.373469 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.373392    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-workload-socket\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.373469 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.373433    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-credential-socket\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.373712 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.373689    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-data\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.373904 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.373882    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istiod-ca-cert\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.375368 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.375347    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-envoy\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.375805 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.375782    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-podinfo\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.385160 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.385141    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t86s6\" (UniqueName: \"kubernetes.io/projected/e232466f-ae92-4709-8a3b-8b1135ef8a9b-kube-api-access-t86s6\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.387691 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.387668    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-token\") pod \"e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") " pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.464426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.464365    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:40.531718 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.531684    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7796-0c72b5" podUID="3ba654ab-f4fa-4892-839b-b3b67e1bcdc4" containerName="test-curl-7796-0c72b5" containerID="cri-o://2e857422a4eb90ebd052a5e77e3b0b7b99c2642ce493da1a400f50f6555cafd4" gracePeriod=2
Sep 10 13:34:40.695416 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.695390    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-4765f4"
Sep 10 13:34:40.751634 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.751556    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"]
Sep 10 13:34:40.754317 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:34:40.754288    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode232466f_ae92_4709_8a3b_8b1135ef8a9b.slice/crio-eed48d150b22978da57619ff1454ff5c044d27ef97eea28bb1a0e952e7742866 WatchSource:0}: Error finding container eed48d150b22978da57619ff1454ff5c044d27ef97eea28bb1a0e952e7742866: Status 404 returned error can't find the container with id eed48d150b22978da57619ff1454ff5c044d27ef97eea28bb1a0e952e7742866
Sep 10 13:34:40.756650 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.756618    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:34:40.756747 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.756684    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:34:40.756747 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.756713    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:34:40.776509 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.776490    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zn4qw\" (UniqueName: \"kubernetes.io/projected/f7855202-35fd-4989-92b5-5026729437f0-kube-api-access-zn4qw\") pod \"f7855202-35fd-4989-92b5-5026729437f0\" (UID: \"f7855202-35fd-4989-92b5-5026729437f0\") "
Sep 10 13:34:40.778411 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.778391    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7855202-35fd-4989-92b5-5026729437f0-kube-api-access-zn4qw" (OuterVolumeSpecName: "kube-api-access-zn4qw") pod "f7855202-35fd-4989-92b5-5026729437f0" (UID: "f7855202-35fd-4989-92b5-5026729437f0"). InnerVolumeSpecName "kube-api-access-zn4qw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:34:40.877537 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:40.877390    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn4qw\" (UniqueName: \"kubernetes.io/projected/f7855202-35fd-4989-92b5-5026729437f0-kube-api-access-zn4qw\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:34:41.539089 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:41.539054    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx" event={"ID":"e232466f-ae92-4709-8a3b-8b1135ef8a9b","Type":"ContainerStarted","Data":"1a62a0ca5587f8e8ff8d5726065b1727344c876317a507af4bdf77c62b1c3a37"}
Sep 10 13:34:41.539612 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:41.539119    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx" event={"ID":"e232466f-ae92-4709-8a3b-8b1135ef8a9b","Type":"ContainerStarted","Data":"eed48d150b22978da57619ff1454ff5c044d27ef97eea28bb1a0e952e7742866"}
Sep 10 13:34:41.540355 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:41.540331    2551 generic.go:356] "Generic (PLEG): container finished" podID="f7855202-35fd-4989-92b5-5026729437f0" containerID="c2bd1a76a00d47a276fab68eef772eb77c135ac8f1176d4c34d031f9d6ed5cbb" exitCode=137
Sep 10 13:34:41.540453 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:41.540381    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-4765f4"
Sep 10 13:34:41.540453 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:41.540400    2551 scope.go:117] "RemoveContainer" containerID="c2bd1a76a00d47a276fab68eef772eb77c135ac8f1176d4c34d031f9d6ed5cbb"
Sep 10 13:34:41.555904 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:41.555884    2551 scope.go:117] "RemoveContainer" containerID="c2bd1a76a00d47a276fab68eef772eb77c135ac8f1176d4c34d031f9d6ed5cbb"
Sep 10 13:34:41.556185 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:34:41.556165    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2bd1a76a00d47a276fab68eef772eb77c135ac8f1176d4c34d031f9d6ed5cbb\": container with ID starting with c2bd1a76a00d47a276fab68eef772eb77c135ac8f1176d4c34d031f9d6ed5cbb not found: ID does not exist" containerID="c2bd1a76a00d47a276fab68eef772eb77c135ac8f1176d4c34d031f9d6ed5cbb"
Sep 10 13:34:41.556249 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:41.556197    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2bd1a76a00d47a276fab68eef772eb77c135ac8f1176d4c34d031f9d6ed5cbb"} err="failed to get container status \"c2bd1a76a00d47a276fab68eef772eb77c135ac8f1176d4c34d031f9d6ed5cbb\": rpc error: code = NotFound desc = could not find container \"c2bd1a76a00d47a276fab68eef772eb77c135ac8f1176d4c34d031f9d6ed5cbb\": container with ID starting with c2bd1a76a00d47a276fab68eef772eb77c135ac8f1176d4c34d031f9d6ed5cbb not found: ID does not exist"
Sep 10 13:34:41.585452 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:34:41.585416    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7796-4765f4\" is forbidden: User \"system:node:ip-10-0-135-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-117.ec2.internal' and this object" podUID="f7855202-35fd-4989-92b5-5026729437f0" pod="opendatahub/test-curl-7796-4765f4"
Sep 10 13:34:41.587373 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:41.587334    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx" podStartSLOduration=1.5873209080000001 podStartE2EDuration="1.587320908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:34:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:34:41.583339749 +0000 UTC m=+992.402378148" watchObservedRunningTime="2026-09-10 13:34:41.587320908 +0000 UTC m=+992.406359307"
Sep 10 13:34:41.713116 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:41.713027    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7855202-35fd-4989-92b5-5026729437f0" path="/var/lib/kubelet/pods/f7855202-35fd-4989-92b5-5026729437f0/volumes"
Sep 10 13:34:42.465480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.465440    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:42.471043 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.471011    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:42.550186 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.550153    2551 generic.go:356] "Generic (PLEG): container finished" podID="3ba654ab-f4fa-4892-839b-b3b67e1bcdc4" containerID="2e857422a4eb90ebd052a5e77e3b0b7b99c2642ce493da1a400f50f6555cafd4" exitCode=137
Sep 10 13:34:42.550911 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.550891    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:42.552075 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.552048    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:34:42.692981 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.692954    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-0c72b5"
Sep 10 13:34:42.735890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.735862    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"]
Sep 10 13:34:42.738592 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.738559    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.744849 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.744827    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3f02bf87-openshift-default-dockercfg-qjxqp\""
Sep 10 13:34:42.770365 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.770338    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"]
Sep 10 13:34:42.795682 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.795655    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jz54b\" (UniqueName: \"kubernetes.io/projected/3ba654ab-f4fa-4892-839b-b3b67e1bcdc4-kube-api-access-jz54b\") pod \"3ba654ab-f4fa-4892-839b-b3b67e1bcdc4\" (UID: \"3ba654ab-f4fa-4892-839b-b3b67e1bcdc4\") "
Sep 10 13:34:42.798311 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.798286    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ba654ab-f4fa-4892-839b-b3b67e1bcdc4-kube-api-access-jz54b" (OuterVolumeSpecName: "kube-api-access-jz54b") pod "3ba654ab-f4fa-4892-839b-b3b67e1bcdc4" (UID: "3ba654ab-f4fa-4892-839b-b3b67e1bcdc4"). InnerVolumeSpecName "kube-api-access-jz54b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:34:42.897615 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.897566    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-credential-socket\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.897790 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.897631    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-workload-certs\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.897790 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.897653    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-workload-socket\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.897790 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.897677    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-podinfo\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.897790 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.897697    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-data\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.897941 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.897798    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-token\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.897941 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.897834    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-envoy\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.897941 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.897848    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istiod-ca-cert\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.897941 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.897865    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54k7x\" (UniqueName: \"kubernetes.io/projected/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-kube-api-access-54k7x\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.897941 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.897919    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz54b\" (UniqueName: \"kubernetes.io/projected/3ba654ab-f4fa-4892-839b-b3b67e1bcdc4-kube-api-access-jz54b\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:34:42.998985 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.998895    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-podinfo\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.998985 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.998933    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-data\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.999213 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.999026    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-token\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.999213 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.999091    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-envoy\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.999213 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.999113    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istiod-ca-cert\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.999213 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.999133    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54k7x\" (UniqueName: \"kubernetes.io/projected/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-kube-api-access-54k7x\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.999431 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.999287    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-data\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.999431 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.999333    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-credential-socket\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.999431 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.999378    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-workload-certs\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.999431 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.999422    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-workload-socket\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.999731 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.999707    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-workload-certs\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.999846 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.999821    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-workload-socket\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.999903 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.999877    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istiod-ca-cert\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:42.999976 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:42.999954    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-credential-socket\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:43.001435 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:43.001413    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-envoy\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:43.001549 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:43.001534    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-podinfo\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:43.009796 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:43.009774    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-token\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:43.014918 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:43.014896    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54k7x\" (UniqueName: \"kubernetes.io/projected/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-kube-api-access-54k7x\") pod \"e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") " pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:43.050213 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:43.050183    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:43.335917 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:43.335891    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"]
Sep 10 13:34:43.337664 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:34:43.337639    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a6e01c0_ce68_497d_b3fb_6fd07a2e75e8.slice/crio-12e76c13adeccf04823f1d851117be3526371eb9b6ded81afc8339e870551573 WatchSource:0}: Error finding container 12e76c13adeccf04823f1d851117be3526371eb9b6ded81afc8339e870551573: Status 404 returned error can't find the container with id 12e76c13adeccf04823f1d851117be3526371eb9b6ded81afc8339e870551573
Sep 10 13:34:43.339829 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:43.339799    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:34:43.339891 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:43.339859    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:34:43.339891 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:43.339886    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:34:43.561602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:43.557980    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7796-0c72b5"
Sep 10 13:34:43.561602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:43.557984    2551 scope.go:117] "RemoveContainer" containerID="2e857422a4eb90ebd052a5e77e3b0b7b99c2642ce493da1a400f50f6555cafd4"
Sep 10 13:34:43.562391 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:43.562002    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt" event={"ID":"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8","Type":"ContainerStarted","Data":"79e7acc2ba9e8ec9d551e709d3d1422511223a8dcd592a61b300b0a8b00ad5f2"}
Sep 10 13:34:43.562391 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:43.562073    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt" event={"ID":"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8","Type":"ContainerStarted","Data":"12e76c13adeccf04823f1d851117be3526371eb9b6ded81afc8339e870551573"}
Sep 10 13:34:43.639516 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:34:43.639424    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7796-0c72b5\" is forbidden: User \"system:node:ip-10-0-135-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-117.ec2.internal' and this object" podUID="3ba654ab-f4fa-4892-839b-b3b67e1bcdc4" pod="opendatahub/test-curl-7796-0c72b5"
Sep 10 13:34:43.710551 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:43.710520    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ba654ab-f4fa-4892-839b-b3b67e1bcdc4" path="/var/lib/kubelet/pods/3ba654ab-f4fa-4892-839b-b3b67e1bcdc4/volumes"
Sep 10 13:34:44.051225 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:44.051195    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:44.055929 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:44.055905    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:44.095939 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:44.095896    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt" podStartSLOduration=2.095881474 podStartE2EDuration="2.095881474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:34:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:34:43.636698245 +0000 UTC m=+994.455736652" watchObservedRunningTime="2026-09-10 13:34:44.095881474 +0000 UTC m=+994.914919873"
Sep 10 13:34:44.566831 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:44.566805    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:44.567730 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:44.567706    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:34:49.831055 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:34:49.831018    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:35:00.607778 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.607726    2551 patch_prober.go:28] interesting pod/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 13:35:00.608170 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.607812    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt" podUID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 13:35:00.647906 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.647873    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"]
Sep 10 13:35:00.653158 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.653134    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.658829 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.658803    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d6159c-openshift-default-dockercfg-tmkb5\""
Sep 10 13:35:00.675338 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.675312    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"]
Sep 10 13:35:00.773054 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.773024    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-envoy\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.773227 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.773062    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a9d58aa-32da-46cb-9b71-deb195f77b43-istiod-ca-cert\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.773227 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.773087    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-data\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.773227 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.773179    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-token\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.773332 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.773273    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-workload-certs\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.773332 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.773304    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-podinfo\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.773406 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.773363    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-workload-socket\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.773449 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.773402    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-credential-socket\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.773493 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.773460    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb5s7\" (UniqueName: \"kubernetes.io/projected/1a9d58aa-32da-46cb-9b71-deb195f77b43-kube-api-access-sb5s7\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.874818 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.874730    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-workload-socket\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.874818 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.874783    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-credential-socket\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.874818 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.874816    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb5s7\" (UniqueName: \"kubernetes.io/projected/1a9d58aa-32da-46cb-9b71-deb195f77b43-kube-api-access-sb5s7\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.875122 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.874894    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-envoy\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.875122 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.874923    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a9d58aa-32da-46cb-9b71-deb195f77b43-istiod-ca-cert\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.875122 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.874956    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-data\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.875122 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.874978    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-token\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.875122 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.875029    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-workload-certs\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.875122 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.875055    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-podinfo\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.875436 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.875306    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-workload-socket\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.875550 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.875517    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-credential-socket\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.875636 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.875534    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-data\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.875636 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.875600    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-workload-certs\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.875743 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.875662    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a9d58aa-32da-46cb-9b71-deb195f77b43-istiod-ca-cert\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.877200 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.877174    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-envoy\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.877519 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.877501    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-podinfo\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.886785 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.886762    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-token\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.887102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.886864    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb5s7\" (UniqueName: \"kubernetes.io/projected/1a9d58aa-32da-46cb-9b71-deb195f77b43-kube-api-access-sb5s7\") pod \"e2e-inf-b-d6159c-openshift-default-754586568-6b69l\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") " pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:00.964544 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:00.964514    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:01.246250 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.246227    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"]
Sep 10 13:35:01.247004 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:35:01.246979    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a9d58aa_32da_46cb_9b71_deb195f77b43.slice/crio-60cb3e32010742eb383122c5c877f97b325c0c25e58483180756378371f81bbe WatchSource:0}: Error finding container 60cb3e32010742eb383122c5c877f97b325c0c25e58483180756378371f81bbe: Status 404 returned error can't find the container with id 60cb3e32010742eb383122c5c877f97b325c0c25e58483180756378371f81bbe
Sep 10 13:35:01.249317 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.249284    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:35:01.249395 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.249346    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:35:01.249395 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.249374    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:35:01.659909 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.659873    2551 generic.go:356] "Generic (PLEG): container finished" podID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" containerID="1a62a0ca5587f8e8ff8d5726065b1727344c876317a507af4bdf77c62b1c3a37" exitCode=0
Sep 10 13:35:01.660384 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.659948    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx" event={"ID":"e232466f-ae92-4709-8a3b-8b1135ef8a9b","Type":"ContainerDied","Data":"1a62a0ca5587f8e8ff8d5726065b1727344c876317a507af4bdf77c62b1c3a37"}
Sep 10 13:35:01.660384 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.660374    2551 scope.go:117] "RemoveContainer" containerID="1a62a0ca5587f8e8ff8d5726065b1727344c876317a507af4bdf77c62b1c3a37"
Sep 10 13:35:01.661612 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.661557    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:35:01.661741 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.661659    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:35:01.661741 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.661699    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:35:01.661878 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.661860    2551 generic.go:356] "Generic (PLEG): container finished" podID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerID="79e7acc2ba9e8ec9d551e709d3d1422511223a8dcd592a61b300b0a8b00ad5f2" exitCode=0
Sep 10 13:35:01.661939 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.661923    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt" event={"ID":"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8","Type":"ContainerDied","Data":"79e7acc2ba9e8ec9d551e709d3d1422511223a8dcd592a61b300b0a8b00ad5f2"}
Sep 10 13:35:01.662309 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.662290    2551 scope.go:117] "RemoveContainer" containerID="79e7acc2ba9e8ec9d551e709d3d1422511223a8dcd592a61b300b0a8b00ad5f2"
Sep 10 13:35:01.663389 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.663358    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:35:01.663488 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.663438    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:35:01.663488 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.663480    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:35:01.663902 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.663880    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l" event={"ID":"1a9d58aa-32da-46cb-9b71-deb195f77b43","Type":"ContainerStarted","Data":"10f20b456f5fb6f71e64518b616528b39ef13d7ef4b58abcb929504dee0414c3"}
Sep 10 13:35:01.664011 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.663927    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l" event={"ID":"1a9d58aa-32da-46cb-9b71-deb195f77b43","Type":"ContainerStarted","Data":"60cb3e32010742eb383122c5c877f97b325c0c25e58483180756378371f81bbe"}
Sep 10 13:35:01.743138 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.742380    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l" podStartSLOduration=1.7423588030000001 podStartE2EDuration="1.742358803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:35:01.741685293 +0000 UTC m=+1012.560723716" watchObservedRunningTime="2026-09-10 13:35:01.742358803 +0000 UTC m=+1012.561397204"
Sep 10 13:35:01.965040 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:01.964939    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:02.674637 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:02.674569    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt" event={"ID":"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8","Type":"ContainerStarted","Data":"5babd8353ba420ea806ee8c137ef6ebc011c26deaafd84771c0e395fdc28d603"}
Sep 10 13:35:02.683136 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:02.683103    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx" event={"ID":"e232466f-ae92-4709-8a3b-8b1135ef8a9b","Type":"ContainerStarted","Data":"0ced79abdfe66b5ffd81993e4432db236afc87a4dc399c41388c16d947e9ee0d"}
Sep 10 13:35:02.977750 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:02.977715    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:03.050806 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:03.050754    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:35:03.051480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:03.050967    2551 patch_prober.go:28] interesting pod/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 10 13:35:03.051480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:03.051171    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt" podUID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 10 13:35:03.465836 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:03.465734    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:35:03.466029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:03.465879    2551 patch_prober.go:28] interesting pod/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 10 13:35:03.466029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:03.465921    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx" podUID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 10 13:35:03.687634 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:03.687604    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:03.690352 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:03.690323    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:04.051404 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:04.051368    2551 patch_prober.go:28] interesting pod/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 10 13:35:04.051753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:04.051709    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt" podUID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 10 13:35:04.465452 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:04.465347    2551 patch_prober.go:28] interesting pod/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 10 13:35:04.465452 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:04.465417    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx" podUID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 10 13:35:05.050998 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:05.050954    2551 patch_prober.go:28] interesting pod/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 10 13:35:05.051505 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:05.051026    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt" podUID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 10 13:35:05.455973 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:05.455871    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:35:05.464910 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:05.464867    2551 patch_prober.go:28] interesting pod/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 10 13:35:05.465107 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:05.464933    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx" podUID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 10 13:35:06.052103 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:06.052065    2551 patch_prober.go:28] interesting pod/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 10 13:35:06.052712 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:06.052679    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt" podUID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 10 13:35:06.465584 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:06.465469    2551 patch_prober.go:28] interesting pod/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 10 13:35:06.465861 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:06.465814    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx" podUID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 10 13:35:07.051114 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:07.051070    2551 patch_prober.go:28] interesting pod/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 10 13:35:07.051431 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:07.051380    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt" podUID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 10 13:35:07.465166 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:07.465066    2551 patch_prober.go:28] interesting pod/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 10 13:35:07.465737 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:07.465703    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx" podUID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 10 13:35:08.054492 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.054456    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:35:08.054922 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.054897    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:35:08.055473 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.055452    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:35:08.272373 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.272341    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"]
Sep 10 13:35:08.274739 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.274721    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.281203 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.281189    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-3bafe380-kserve-self-signed-certs\""
Sep 10 13:35:08.292871 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.292850    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"]
Sep 10 13:35:08.354863 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.354780    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-home\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.354863 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.354816    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a4e2303-9505-473f-8ae1-8d0e68355ebb-tls-certs\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.354863 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.354837    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-tmp-dir\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.355129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.354931    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-model-cache\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.355129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.355021    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtbgz\" (UniqueName: \"kubernetes.io/projected/0a4e2303-9505-473f-8ae1-8d0e68355ebb-kube-api-access-wtbgz\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.355129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.355092    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-dshm\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.456156 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.456119    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtbgz\" (UniqueName: \"kubernetes.io/projected/0a4e2303-9505-473f-8ae1-8d0e68355ebb-kube-api-access-wtbgz\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.456308 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.456184    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-dshm\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.456308 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.456252    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-home\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.456308 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.456281    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a4e2303-9505-473f-8ae1-8d0e68355ebb-tls-certs\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.456501 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.456307    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-tmp-dir\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.456501 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.456364    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-model-cache\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.456636 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.456604    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-home\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.456781 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.456761    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-model-cache\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.456841 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.456782    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-tmp-dir\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.458440 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.458425    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-dshm\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.458799 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.458781    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a4e2303-9505-473f-8ae1-8d0e68355ebb-tls-certs\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.466878 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.466857    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtbgz\" (UniqueName: \"kubernetes.io/projected/0a4e2303-9505-473f-8ae1-8d0e68355ebb-kube-api-access-wtbgz\") pod \"e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") " pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.469053 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.469034    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:35:08.469430 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.469414    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:35:08.470029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.470012    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:35:08.583507 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.583476    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:08.883909 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:08.883885    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"]
Sep 10 13:35:08.886309 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:35:08.886248    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a4e2303_9505_473f_8ae1_8d0e68355ebb.slice/crio-daadef57d34a4765ffcc1ed1b0ec963fbbdf6ec2df6c0ac1926d2d73a3b64205 WatchSource:0}: Error finding container daadef57d34a4765ffcc1ed1b0ec963fbbdf6ec2df6c0ac1926d2d73a3b64205: Status 404 returned error can't find the container with id daadef57d34a4765ffcc1ed1b0ec963fbbdf6ec2df6c0ac1926d2d73a3b64205
Sep 10 13:35:09.726632 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:09.726602    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh" event={"ID":"0a4e2303-9505-473f-8ae1-8d0e68355ebb","Type":"ContainerStarted","Data":"c1a38d2d9fef3e39ca49e12b7cc9acae133620081be9abe7dab396cdb5fe7846"}
Sep 10 13:35:09.726993 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:09.726643    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh" event={"ID":"0a4e2303-9505-473f-8ae1-8d0e68355ebb","Type":"ContainerStarted","Data":"daadef57d34a4765ffcc1ed1b0ec963fbbdf6ec2df6c0ac1926d2d73a3b64205"}
Sep 10 13:35:09.748621 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:09.748567    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh" podStartSLOduration=1.7485521149999999 podStartE2EDuration="1.748552115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:35:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:35:09.746965959 +0000 UTC m=+1020.566004360" watchObservedRunningTime="2026-09-10 13:35:09.748552115 +0000 UTC m=+1020.567590515"
Sep 10 13:35:18.584309 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:18.584266    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:18.584721 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:18.584323    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:18.596897 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:18.596869    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:18.792008 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:18.791980    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:35:22.498086 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:22.498048    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:35:25.817616 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:25.817554    2551 generic.go:356] "Generic (PLEG): container finished" podID="1a9d58aa-32da-46cb-9b71-deb195f77b43" containerID="10f20b456f5fb6f71e64518b616528b39ef13d7ef4b58abcb929504dee0414c3" exitCode=0
Sep 10 13:35:25.818015 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:25.817634    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l" event={"ID":"1a9d58aa-32da-46cb-9b71-deb195f77b43","Type":"ContainerDied","Data":"10f20b456f5fb6f71e64518b616528b39ef13d7ef4b58abcb929504dee0414c3"}
Sep 10 13:35:25.818060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:25.818049    2551 scope.go:117] "RemoveContainer" containerID="10f20b456f5fb6f71e64518b616528b39ef13d7ef4b58abcb929504dee0414c3"
Sep 10 13:35:25.819176 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:25.819148    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:35:25.819239 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:25.819221    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:35:25.819277 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:25.819259    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:35:26.824530 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:26.824499    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l" event={"ID":"1a9d58aa-32da-46cb-9b71-deb195f77b43","Type":"ContainerStarted","Data":"1e7a46e1b84866cae0537715acc1801ce8a1d8b4c788188e2f1ad285f78d93bf"}
Sep 10 13:35:26.965713 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:26.965671    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:26.965923 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:26.965841    2551 patch_prober.go:28] interesting pod/e2e-inf-b-d6159c-openshift-default-754586568-6b69l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 10 13:35:26.965923 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:26.965893    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l" podUID="1a9d58aa-32da-46cb-9b71-deb195f77b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 10 13:35:27.965225 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:27.965181    2551 patch_prober.go:28] interesting pod/e2e-inf-b-d6159c-openshift-default-754586568-6b69l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 10 13:35:27.965722 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:27.965264    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l" podUID="1a9d58aa-32da-46cb-9b71-deb195f77b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 10 13:35:28.964972 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:28.964933    2551 patch_prober.go:28] interesting pod/e2e-inf-b-d6159c-openshift-default-754586568-6b69l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 10 13:35:28.965271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:28.965242    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l" podUID="1a9d58aa-32da-46cb-9b71-deb195f77b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 10 13:35:29.024314 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:29.024278    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"]
Sep 10 13:35:29.024561 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:29.024529    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt" podUID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerName="istio-proxy" containerID="cri-o://5babd8353ba420ea806ee8c137ef6ebc011c26deaafd84771c0e395fdc28d603" gracePeriod=30
Sep 10 13:35:29.965382 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:29.965343    2551 patch_prober.go:28] interesting pod/e2e-inf-b-d6159c-openshift-default-754586568-6b69l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 10 13:35:29.965911 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:29.965415    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l" podUID="1a9d58aa-32da-46cb-9b71-deb195f77b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 10 13:35:30.964962 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:30.964918    2551 patch_prober.go:28] interesting pod/e2e-inf-b-d6159c-openshift-default-754586568-6b69l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 10 13:35:30.965173 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:30.964986    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l" podUID="1a9d58aa-32da-46cb-9b71-deb195f77b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 10 13:35:32.087999 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:32.087968    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:32.088471 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:32.088355    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:32.089606 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:32.089563    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:35:34.271555 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.271522    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:35:34.342106 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342043    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istiod-ca-cert\") pod \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") "
Sep 10 13:35:34.342225 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342106    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54k7x\" (UniqueName: \"kubernetes.io/projected/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-kube-api-access-54k7x\") pod \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") "
Sep 10 13:35:34.342225 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342154    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-token\") pod \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") "
Sep 10 13:35:34.342225 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342177    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-podinfo\") pod \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") "
Sep 10 13:35:34.342350 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342238    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-workload-socket\") pod \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") "
Sep 10 13:35:34.342350 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342275    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-workload-certs\") pod \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") "
Sep 10 13:35:34.342350 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342319    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-data\") pod \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") "
Sep 10 13:35:34.342350 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342343    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-envoy\") pod \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") "
Sep 10 13:35:34.342553 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342370    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-credential-socket\") pod \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\" (UID: \"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8\") "
Sep 10 13:35:34.342635 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342600    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" (UID: "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:35:34.342635 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342626    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" (UID: "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:35:34.342750 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342707    2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-workload-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:35:34.342750 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342727    2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-data\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:35:34.342976 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342950    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" (UID: "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:35:34.343058 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.342988    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" (UID: "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:35:34.343058 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.343015    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" (UID: "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:35:34.344525 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.344503    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-kube-api-access-54k7x" (OuterVolumeSpecName: "kube-api-access-54k7x") pod "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" (UID: "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8"). InnerVolumeSpecName "kube-api-access-54k7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:35:34.344525 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.344508    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" (UID: "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:35:34.344766 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.344751    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" (UID: "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:35:34.344925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.344904    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" (UID: "3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 13:35:34.443480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.443451    2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-workload-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:35:34.443480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.443475    2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-envoy\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:35:34.443480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.443484    2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-credential-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:35:34.443720 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.443492    2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istiod-ca-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:35:34.443720 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.443502    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54k7x\" (UniqueName: \"kubernetes.io/projected/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-kube-api-access-54k7x\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:35:34.443720 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.443509    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-token\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:35:34.443720 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.443517    2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8-istio-podinfo\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:35:34.864442 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.864402    2551 generic.go:356] "Generic (PLEG): container finished" podID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerID="5babd8353ba420ea806ee8c137ef6ebc011c26deaafd84771c0e395fdc28d603" exitCode=0
Sep 10 13:35:34.864649 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.864456    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"
Sep 10 13:35:34.864649 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.864489    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt" event={"ID":"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8","Type":"ContainerDied","Data":"5babd8353ba420ea806ee8c137ef6ebc011c26deaafd84771c0e395fdc28d603"}
Sep 10 13:35:34.864649 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.864548    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt" event={"ID":"3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8","Type":"ContainerDied","Data":"12e76c13adeccf04823f1d851117be3526371eb9b6ded81afc8339e870551573"}
Sep 10 13:35:34.864649 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.864599    2551 scope.go:117] "RemoveContainer" containerID="5babd8353ba420ea806ee8c137ef6ebc011c26deaafd84771c0e395fdc28d603"
Sep 10 13:35:34.874269 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.874251    2551 scope.go:117] "RemoveContainer" containerID="79e7acc2ba9e8ec9d551e709d3d1422511223a8dcd592a61b300b0a8b00ad5f2"
Sep 10 13:35:34.882098 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.882077    2551 scope.go:117] "RemoveContainer" containerID="5babd8353ba420ea806ee8c137ef6ebc011c26deaafd84771c0e395fdc28d603"
Sep 10 13:35:34.882380 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:35:34.882361    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5babd8353ba420ea806ee8c137ef6ebc011c26deaafd84771c0e395fdc28d603\": container with ID starting with 5babd8353ba420ea806ee8c137ef6ebc011c26deaafd84771c0e395fdc28d603 not found: ID does not exist" containerID="5babd8353ba420ea806ee8c137ef6ebc011c26deaafd84771c0e395fdc28d603"
Sep 10 13:35:34.882429 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.882388    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5babd8353ba420ea806ee8c137ef6ebc011c26deaafd84771c0e395fdc28d603"} err="failed to get container status \"5babd8353ba420ea806ee8c137ef6ebc011c26deaafd84771c0e395fdc28d603\": rpc error: code = NotFound desc = could not find container \"5babd8353ba420ea806ee8c137ef6ebc011c26deaafd84771c0e395fdc28d603\": container with ID starting with 5babd8353ba420ea806ee8c137ef6ebc011c26deaafd84771c0e395fdc28d603 not found: ID does not exist"
Sep 10 13:35:34.882429 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.882406    2551 scope.go:117] "RemoveContainer" containerID="79e7acc2ba9e8ec9d551e709d3d1422511223a8dcd592a61b300b0a8b00ad5f2"
Sep 10 13:35:34.882708 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:35:34.882690    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79e7acc2ba9e8ec9d551e709d3d1422511223a8dcd592a61b300b0a8b00ad5f2\": container with ID starting with 79e7acc2ba9e8ec9d551e709d3d1422511223a8dcd592a61b300b0a8b00ad5f2 not found: ID does not exist" containerID="79e7acc2ba9e8ec9d551e709d3d1422511223a8dcd592a61b300b0a8b00ad5f2"
Sep 10 13:35:34.882760 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.882713    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79e7acc2ba9e8ec9d551e709d3d1422511223a8dcd592a61b300b0a8b00ad5f2"} err="failed to get container status \"79e7acc2ba9e8ec9d551e709d3d1422511223a8dcd592a61b300b0a8b00ad5f2\": rpc error: code = NotFound desc = could not find container \"79e7acc2ba9e8ec9d551e709d3d1422511223a8dcd592a61b300b0a8b00ad5f2\": container with ID starting with 79e7acc2ba9e8ec9d551e709d3d1422511223a8dcd592a61b300b0a8b00ad5f2 not found: ID does not exist"
Sep 10 13:35:34.889608 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.889585    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"]
Sep 10 13:35:34.893144 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:34.893125    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3f02bf87-openshift-default-865d8679b8-fw8pt"]
Sep 10 13:35:35.332829 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:35.332793    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:35:35.710267 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:35.710189    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" path="/var/lib/kubelet/pods/3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8/volumes"
Sep 10 13:35:55.229253 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:55.229213    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:35:58.073880 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.073846    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd"]
Sep 10 13:35:58.074258 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.074193    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerName="istio-proxy"
Sep 10 13:35:58.074258 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.074204    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerName="istio-proxy"
Sep 10 13:35:58.074258 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.074220    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerName="istio-proxy"
Sep 10 13:35:58.074258 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.074226    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerName="istio-proxy"
Sep 10 13:35:58.074443 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.074283    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerName="istio-proxy"
Sep 10 13:35:58.076699 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.076684    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd"
Sep 10 13:35:58.081365 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.081343    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:35:58.081523 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.081509    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:35:58.090008 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.089988    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd"]
Sep 10 13:35:58.159943 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.159912    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3523e2ed-2818-45fd-81d2-f22707476e22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd\" (UID: \"3523e2ed-2818-45fd-81d2-f22707476e22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd"
Sep 10 13:35:58.260742 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.260711    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3523e2ed-2818-45fd-81d2-f22707476e22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd\" (UID: \"3523e2ed-2818-45fd-81d2-f22707476e22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd"
Sep 10 13:35:58.261283 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.261264    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3523e2ed-2818-45fd-81d2-f22707476e22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd\" (UID: \"3523e2ed-2818-45fd-81d2-f22707476e22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd"
Sep 10 13:35:58.387067 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.387004    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd"
Sep 10 13:35:58.691953 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.691930    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd"]
Sep 10 13:35:58.694019 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:35:58.693993    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3523e2ed_2818_45fd_81d2_f22707476e22.slice/crio-5093b81dd2d610f2b02bf71717a4cfbd561be7e17570983b732a055f5cc7ce1c WatchSource:0}: Error finding container 5093b81dd2d610f2b02bf71717a4cfbd561be7e17570983b732a055f5cc7ce1c: Status 404 returned error can't find the container with id 5093b81dd2d610f2b02bf71717a4cfbd561be7e17570983b732a055f5cc7ce1c
Sep 10 13:35:58.696095 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.696076    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 13:35:58.989546 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:35:58.989509    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd" event={"ID":"3523e2ed-2818-45fd-81d2-f22707476e22","Type":"ContainerStarted","Data":"5093b81dd2d610f2b02bf71717a4cfbd561be7e17570983b732a055f5cc7ce1c"}
Sep 10 13:36:00.580396 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.580360    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"]
Sep 10 13:36:00.580995 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.580974    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a6e01c0-ce68-497d-b3fb-6fd07a2e75e8" containerName="istio-proxy"
Sep 10 13:36:00.583098 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.583074    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.586513 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.586492    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-3bafe380-kserve-self-signed-certs\""
Sep 10 13:36:00.598646 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.598624    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"]
Sep 10 13:36:00.684326 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.684288    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-home\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.684326 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.684330    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-model-cache\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.684539 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.684424    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94bed86a-e71e-4b8b-9db6-517ac1351dd8-tls-certs\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.684539 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.684506    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8bbk\" (UniqueName: \"kubernetes.io/projected/94bed86a-e71e-4b8b-9db6-517ac1351dd8-kube-api-access-b8bbk\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.684654 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.684559    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-tmp-dir\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.684654 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.684624    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-dshm\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.785539 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.785500    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-tmp-dir\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.785737 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.785562    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-dshm\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.785737 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.785639    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-home\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.785737 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.785668    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-model-cache\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.785906 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.785738    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94bed86a-e71e-4b8b-9db6-517ac1351dd8-tls-certs\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.785906 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.785802    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8bbk\" (UniqueName: \"kubernetes.io/projected/94bed86a-e71e-4b8b-9db6-517ac1351dd8-kube-api-access-b8bbk\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.786029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.785947    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-tmp-dir\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.786029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.786008    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-home\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.786140 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.786110    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-model-cache\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.788061 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.788034    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-dshm\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.788362 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.788336    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94bed86a-e71e-4b8b-9db6-517ac1351dd8-tls-certs\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.795780 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.795753    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8bbk\" (UniqueName: \"kubernetes.io/projected/94bed86a-e71e-4b8b-9db6-517ac1351dd8-kube-api-access-b8bbk\") pod \"e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") " pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:00.893786 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:00.893714    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:01.518255 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:01.518192    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"]
Sep 10 13:36:01.525914 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:36:01.525884    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94bed86a_e71e_4b8b_9db6_517ac1351dd8.slice/crio-153a71ef5ab77dc523c9d9c244ea204e6affc8736a85749fbd38f6a5ef10face WatchSource:0}: Error finding container 153a71ef5ab77dc523c9d9c244ea204e6affc8736a85749fbd38f6a5ef10face: Status 404 returned error can't find the container with id 153a71ef5ab77dc523c9d9c244ea204e6affc8736a85749fbd38f6a5ef10face
Sep 10 13:36:02.006444 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:02.006413    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj" event={"ID":"94bed86a-e71e-4b8b-9db6-517ac1351dd8","Type":"ContainerStarted","Data":"45e469762a6aec4c9f4686373eb9dc29a8e9a944bdaa0ee5c6ef192f8535d064"}
Sep 10 13:36:02.006828 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:02.006464    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj" event={"ID":"94bed86a-e71e-4b8b-9db6-517ac1351dd8","Type":"ContainerStarted","Data":"153a71ef5ab77dc523c9d9c244ea204e6affc8736a85749fbd38f6a5ef10face"}
Sep 10 13:36:02.008044 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:02.008018    2551 generic.go:356] "Generic (PLEG): container finished" podID="3523e2ed-2818-45fd-81d2-f22707476e22" containerID="42008e199cb3b74a1a4c7d7ed2ba9d9c9a497173d1bb12fd9118deb40c3fc844" exitCode=0
Sep 10 13:36:02.008148 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:02.008086    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd" event={"ID":"3523e2ed-2818-45fd-81d2-f22707476e22","Type":"ContainerDied","Data":"42008e199cb3b74a1a4c7d7ed2ba9d9c9a497173d1bb12fd9118deb40c3fc844"}
Sep 10 13:36:02.027831 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:02.027786    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj" podStartSLOduration=2.027773511 podStartE2EDuration="2.027773511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:02.026666216 +0000 UTC m=+1072.845704617" watchObservedRunningTime="2026-09-10 13:36:02.027773511 +0000 UTC m=+1072.846811910"
Sep 10 13:36:03.141384 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:03.141359    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd"
Sep 10 13:36:03.309240 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:03.309162    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3523e2ed-2818-45fd-81d2-f22707476e22-maas-api-service-ca\") pod \"3523e2ed-2818-45fd-81d2-f22707476e22\" (UID: \"3523e2ed-2818-45fd-81d2-f22707476e22\") "
Sep 10 13:36:03.309491 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:03.309466    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3523e2ed-2818-45fd-81d2-f22707476e22-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3523e2ed-2818-45fd-81d2-f22707476e22" (UID: "3523e2ed-2818-45fd-81d2-f22707476e22"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:36:03.410042 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:03.410013    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3523e2ed-2818-45fd-81d2-f22707476e22-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:04.018464 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.018430    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd" event={"ID":"3523e2ed-2818-45fd-81d2-f22707476e22","Type":"ContainerDied","Data":"5093b81dd2d610f2b02bf71717a4cfbd561be7e17570983b732a055f5cc7ce1c"}
Sep 10 13:36:04.018666 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.018483    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5093b81dd2d610f2b02bf71717a4cfbd561be7e17570983b732a055f5cc7ce1c"
Sep 10 13:36:04.018666 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.018442    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd"
Sep 10 13:36:04.080852 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.080822    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2"]
Sep 10 13:36:04.081273 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.081254    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3523e2ed-2818-45fd-81d2-f22707476e22" containerName="revoke-keys"
Sep 10 13:36:04.081362 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.081277    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3523e2ed-2818-45fd-81d2-f22707476e22" containerName="revoke-keys"
Sep 10 13:36:04.081417 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.081376    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3523e2ed-2818-45fd-81d2-f22707476e22" containerName="revoke-keys"
Sep 10 13:36:04.083308 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.083289    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2"
Sep 10 13:36:04.086834 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.086811    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:36:04.087743 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.087725    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:36:04.097355 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.097331    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2"]
Sep 10 13:36:04.216475 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.216438    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/088cef7f-341c-4163-839d-700b9e8c99f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-deba50-t55w2\" (UID: \"088cef7f-341c-4163-839d-700b9e8c99f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2"
Sep 10 13:36:04.317534 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.317455    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/088cef7f-341c-4163-839d-700b9e8c99f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-deba50-t55w2\" (UID: \"088cef7f-341c-4163-839d-700b9e8c99f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2"
Sep 10 13:36:04.318027 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.318008    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/088cef7f-341c-4163-839d-700b9e8c99f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-deba50-t55w2\" (UID: \"088cef7f-341c-4163-839d-700b9e8c99f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2"
Sep 10 13:36:04.392894 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.392869    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2"
Sep 10 13:36:04.676078 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:04.676054    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2"]
Sep 10 13:36:04.677951 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:36:04.677921    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod088cef7f_341c_4163_839d_700b9e8c99f8.slice/crio-975b5eb1d56063216d6fa802af481fba244acbf4b884570ecda5fb01f926d8f2 WatchSource:0}: Error finding container 975b5eb1d56063216d6fa802af481fba244acbf4b884570ecda5fb01f926d8f2: Status 404 returned error can't find the container with id 975b5eb1d56063216d6fa802af481fba244acbf4b884570ecda5fb01f926d8f2
Sep 10 13:36:05.025246 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:05.025214    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2" event={"ID":"088cef7f-341c-4163-839d-700b9e8c99f8","Type":"ContainerStarted","Data":"bcc6077a13acdc6ecde7daed89d47ef3df54cf54bac1a11ce8c5f037d16b3aa0"}
Sep 10 13:36:05.025363 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:05.025266    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2" event={"ID":"088cef7f-341c-4163-839d-700b9e8c99f8","Type":"ContainerStarted","Data":"975b5eb1d56063216d6fa802af481fba244acbf4b884570ecda5fb01f926d8f2"}
Sep 10 13:36:05.048721 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:05.048670    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2" podStartSLOduration=1.048649916 podStartE2EDuration="1.048649916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:05.046703143 +0000 UTC m=+1075.865741543" watchObservedRunningTime="2026-09-10 13:36:05.048649916 +0000 UTC m=+1075.867688317"
Sep 10 13:36:06.034186 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:06.034139    2551 generic.go:356] "Generic (PLEG): container finished" podID="088cef7f-341c-4163-839d-700b9e8c99f8" containerID="bcc6077a13acdc6ecde7daed89d47ef3df54cf54bac1a11ce8c5f037d16b3aa0" exitCode=0
Sep 10 13:36:06.034595 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:06.034195    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2" event={"ID":"088cef7f-341c-4163-839d-700b9e8c99f8","Type":"ContainerDied","Data":"bcc6077a13acdc6ecde7daed89d47ef3df54cf54bac1a11ce8c5f037d16b3aa0"}
Sep 10 13:36:07.183613 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:07.182568    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2"
Sep 10 13:36:07.346464 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:07.346383    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/088cef7f-341c-4163-839d-700b9e8c99f8-maas-api-service-ca\") pod \"088cef7f-341c-4163-839d-700b9e8c99f8\" (UID: \"088cef7f-341c-4163-839d-700b9e8c99f8\") "
Sep 10 13:36:07.346752 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:07.346724    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/088cef7f-341c-4163-839d-700b9e8c99f8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "088cef7f-341c-4163-839d-700b9e8c99f8" (UID: "088cef7f-341c-4163-839d-700b9e8c99f8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:36:07.447323 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:07.447290    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/088cef7f-341c-4163-839d-700b9e8c99f8-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:08.047677 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:08.047650    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2"
Sep 10 13:36:08.047841 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:08.047647    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2" event={"ID":"088cef7f-341c-4163-839d-700b9e8c99f8","Type":"ContainerDied","Data":"975b5eb1d56063216d6fa802af481fba244acbf4b884570ecda5fb01f926d8f2"}
Sep 10 13:36:08.047841 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:08.047798    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="975b5eb1d56063216d6fa802af481fba244acbf4b884570ecda5fb01f926d8f2"
Sep 10 13:36:10.894540 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:10.894502    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:10.894953 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:10.894555    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:10.906985 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:10.906962    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:11.073376 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:11.073346    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:15.071790 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:15.071757    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd"]
Sep 10 13:36:15.075901 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:15.075877    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4907abfd-qcrgd"]
Sep 10 13:36:15.322916 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:15.322835    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2"]
Sep 10 13:36:15.331350 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:15.331325    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-deba50-t55w2"]
Sep 10 13:36:15.710604 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:15.710495    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="088cef7f-341c-4163-839d-700b9e8c99f8" path="/var/lib/kubelet/pods/088cef7f-341c-4163-839d-700b9e8c99f8/volumes"
Sep 10 13:36:15.710979 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:15.710957    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3523e2ed-2818-45fd-81d2-f22707476e22" path="/var/lib/kubelet/pods/3523e2ed-2818-45fd-81d2-f22707476e22/volumes"
Sep 10 13:36:17.114125 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:17.114091    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:36:25.124305 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.124273    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"]
Sep 10 13:36:25.124731 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.124485    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx" podUID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" containerName="istio-proxy" containerID="cri-o://0ced79abdfe66b5ffd81993e4432db236afc87a4dc399c41388c16d947e9ee0d" gracePeriod=30
Sep 10 13:36:25.822540 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.822508    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"]
Sep 10 13:36:25.822890 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.822876    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="088cef7f-341c-4163-839d-700b9e8c99f8" containerName="revoke-keys"
Sep 10 13:36:25.822947 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.822891    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="088cef7f-341c-4163-839d-700b9e8c99f8" containerName="revoke-keys"
Sep 10 13:36:25.822985 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.822953    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="088cef7f-341c-4163-839d-700b9e8c99f8" containerName="revoke-keys"
Sep 10 13:36:25.826190 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.826171    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:25.829253 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.829233    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-3c32b1-openshift-default-dockercfg-vd5p9\""
Sep 10 13:36:25.845399 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.845381    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"]
Sep 10 13:36:25.918869 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.918833    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8612a427-bc5b-4ecd-9ba1-2c9399912173-istiod-ca-cert\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:25.919050 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.918918    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-token\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:25.919050 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.918953    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-workload-socket\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:25.919050 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.918989    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-data\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:25.919050 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.919017    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h75wp\" (UniqueName: \"kubernetes.io/projected/8612a427-bc5b-4ecd-9ba1-2c9399912173-kube-api-access-h75wp\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:25.919221 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.919089    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-envoy\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:25.919221 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.919149    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-workload-certs\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:25.919221 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.919189    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-credential-socket\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:25.919347 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:25.919219    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-podinfo\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.020707 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.020669    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-workload-certs\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.020891 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.020713    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-credential-socket\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.020891 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.020767    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-podinfo\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.020891 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.020857    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8612a427-bc5b-4ecd-9ba1-2c9399912173-istiod-ca-cert\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.021076 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.020914    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-token\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.021076 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.020942    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-workload-socket\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.021076 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.020982    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-data\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.021230 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.021096    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h75wp\" (UniqueName: \"kubernetes.io/projected/8612a427-bc5b-4ecd-9ba1-2c9399912173-kube-api-access-h75wp\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.021230 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.021105    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-credential-socket\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.021230 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.021142    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-envoy\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.021230 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.021151    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-workload-certs\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.021436 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.021323    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-data\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.021478 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.021434    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-workload-socket\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.021602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.021561    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8612a427-bc5b-4ecd-9ba1-2c9399912173-istiod-ca-cert\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.023197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.023172    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-podinfo\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.023284 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.023172    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-envoy\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.031083 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.031060    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-token\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.031359 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.031338    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h75wp\" (UniqueName: \"kubernetes.io/projected/8612a427-bc5b-4ecd-9ba1-2c9399912173-kube-api-access-h75wp\") pod \"e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") " pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.136623 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.136537    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:26.425504 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.425473    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"]
Sep 10 13:36:26.426537 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:36:26.426512    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8612a427_bc5b_4ecd_9ba1_2c9399912173.slice/crio-ca97d5203c2a355e961e5d7d953d22aa4295086ba9cc813477525b0a6d4eb10d WatchSource:0}: Error finding container ca97d5203c2a355e961e5d7d953d22aa4295086ba9cc813477525b0a6d4eb10d: Status 404 returned error can't find the container with id ca97d5203c2a355e961e5d7d953d22aa4295086ba9cc813477525b0a6d4eb10d
Sep 10 13:36:26.429043 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.429003    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:36:26.429137 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.429088    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:36:26.429180 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:26.429131    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:36:27.142429 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:27.142397    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw" event={"ID":"8612a427-bc5b-4ecd-9ba1-2c9399912173","Type":"ContainerStarted","Data":"e90b849bdb8642547685321ca1798d01d02f07570802ca55136fc92cb14b51b1"}
Sep 10 13:36:27.142893 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:27.142460    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw" event={"ID":"8612a427-bc5b-4ecd-9ba1-2c9399912173","Type":"ContainerStarted","Data":"ca97d5203c2a355e961e5d7d953d22aa4295086ba9cc813477525b0a6d4eb10d"}
Sep 10 13:36:27.176319 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:27.176259    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw" podStartSLOduration=2.176239396 podStartE2EDuration="2.176239396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:27.174118741 +0000 UTC m=+1097.993157143" watchObservedRunningTime="2026-09-10 13:36:27.176239396 +0000 UTC m=+1097.995277799"
Sep 10 13:36:28.136968 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:28.136934    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:28.141426 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:28.141401    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:28.148191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:28.148167    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:28.148866 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:28.148846    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:28.687261 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:28.687228    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:36:29.071163 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:29.071129    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"]
Sep 10 13:36:30.386429 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.386373    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:36:30.470280 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.470248    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-podinfo\") pod \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") "
Sep 10 13:36:30.470280 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.470282    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-token\") pod \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") "
Sep 10 13:36:30.470550 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.470309    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-credential-socket\") pod \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") "
Sep 10 13:36:30.470550 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.470351    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t86s6\" (UniqueName: \"kubernetes.io/projected/e232466f-ae92-4709-8a3b-8b1135ef8a9b-kube-api-access-t86s6\") pod \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") "
Sep 10 13:36:30.470550 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.470426    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-workload-certs\") pod \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") "
Sep 10 13:36:30.470550 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.470486    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istiod-ca-cert\") pod \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") "
Sep 10 13:36:30.470550 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.470512    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-data\") pod \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") "
Sep 10 13:36:30.470861 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.470554    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-envoy\") pod \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") "
Sep 10 13:36:30.470861 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.470608    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-workload-socket\") pod \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\" (UID: \"e232466f-ae92-4709-8a3b-8b1135ef8a9b\") "
Sep 10 13:36:30.470861 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.470847    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e232466f-ae92-4709-8a3b-8b1135ef8a9b" (UID: "e232466f-ae92-4709-8a3b-8b1135ef8a9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:36:30.471048 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.471001    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-data" (OuterVolumeSpecName: "istio-data") pod "e232466f-ae92-4709-8a3b-8b1135ef8a9b" (UID: "e232466f-ae92-4709-8a3b-8b1135ef8a9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:36:30.471048 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.471034    2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-workload-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:30.471425 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.471379    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e232466f-ae92-4709-8a3b-8b1135ef8a9b" (UID: "e232466f-ae92-4709-8a3b-8b1135ef8a9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:36:30.471630 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.471491    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e232466f-ae92-4709-8a3b-8b1135ef8a9b" (UID: "e232466f-ae92-4709-8a3b-8b1135ef8a9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:36:30.471789 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.471757    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e232466f-ae92-4709-8a3b-8b1135ef8a9b" (UID: "e232466f-ae92-4709-8a3b-8b1135ef8a9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:36:30.472904 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.472854    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e232466f-ae92-4709-8a3b-8b1135ef8a9b" (UID: "e232466f-ae92-4709-8a3b-8b1135ef8a9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 13:36:30.473526 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.473501    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-token" (OuterVolumeSpecName: "istio-token") pod "e232466f-ae92-4709-8a3b-8b1135ef8a9b" (UID: "e232466f-ae92-4709-8a3b-8b1135ef8a9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:36:30.473641 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.473564    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e232466f-ae92-4709-8a3b-8b1135ef8a9b" (UID: "e232466f-ae92-4709-8a3b-8b1135ef8a9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:36:30.473857 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.473833    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e232466f-ae92-4709-8a3b-8b1135ef8a9b-kube-api-access-t86s6" (OuterVolumeSpecName: "kube-api-access-t86s6") pod "e232466f-ae92-4709-8a3b-8b1135ef8a9b" (UID: "e232466f-ae92-4709-8a3b-8b1135ef8a9b"). InnerVolumeSpecName "kube-api-access-t86s6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:36:30.571752 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.571721    2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istiod-ca-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:30.571752 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.571748    2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-data\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:30.571954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.571760    2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-envoy\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:30.571954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.571772    2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-workload-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:30.571954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.571783    2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-podinfo\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:30.571954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.571798    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e232466f-ae92-4709-8a3b-8b1135ef8a9b-istio-token\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:30.571954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.571810    2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e232466f-ae92-4709-8a3b-8b1135ef8a9b-credential-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:30.571954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:30.571825    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t86s6\" (UniqueName: \"kubernetes.io/projected/e232466f-ae92-4709-8a3b-8b1135ef8a9b-kube-api-access-t86s6\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:31.165138 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:31.165105    2551 generic.go:356] "Generic (PLEG): container finished" podID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" containerID="0ced79abdfe66b5ffd81993e4432db236afc87a4dc399c41388c16d947e9ee0d" exitCode=0
Sep 10 13:36:31.165332 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:31.165153    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx" event={"ID":"e232466f-ae92-4709-8a3b-8b1135ef8a9b","Type":"ContainerDied","Data":"0ced79abdfe66b5ffd81993e4432db236afc87a4dc399c41388c16d947e9ee0d"}
Sep 10 13:36:31.165332 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:31.165163    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"
Sep 10 13:36:31.165332 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:31.165193    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx" event={"ID":"e232466f-ae92-4709-8a3b-8b1135ef8a9b","Type":"ContainerDied","Data":"eed48d150b22978da57619ff1454ff5c044d27ef97eea28bb1a0e952e7742866"}
Sep 10 13:36:31.165332 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:31.165216    2551 scope.go:117] "RemoveContainer" containerID="0ced79abdfe66b5ffd81993e4432db236afc87a4dc399c41388c16d947e9ee0d"
Sep 10 13:36:31.174293 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:31.174278    2551 scope.go:117] "RemoveContainer" containerID="1a62a0ca5587f8e8ff8d5726065b1727344c876317a507af4bdf77c62b1c3a37"
Sep 10 13:36:31.182057 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:31.182037    2551 scope.go:117] "RemoveContainer" containerID="0ced79abdfe66b5ffd81993e4432db236afc87a4dc399c41388c16d947e9ee0d"
Sep 10 13:36:31.182329 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:36:31.182311    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ced79abdfe66b5ffd81993e4432db236afc87a4dc399c41388c16d947e9ee0d\": container with ID starting with 0ced79abdfe66b5ffd81993e4432db236afc87a4dc399c41388c16d947e9ee0d not found: ID does not exist" containerID="0ced79abdfe66b5ffd81993e4432db236afc87a4dc399c41388c16d947e9ee0d"
Sep 10 13:36:31.182375 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:31.182336    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ced79abdfe66b5ffd81993e4432db236afc87a4dc399c41388c16d947e9ee0d"} err="failed to get container status \"0ced79abdfe66b5ffd81993e4432db236afc87a4dc399c41388c16d947e9ee0d\": rpc error: code = NotFound desc = could not find container \"0ced79abdfe66b5ffd81993e4432db236afc87a4dc399c41388c16d947e9ee0d\": container with ID starting with 0ced79abdfe66b5ffd81993e4432db236afc87a4dc399c41388c16d947e9ee0d not found: ID does not exist"
Sep 10 13:36:31.182375 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:31.182353    2551 scope.go:117] "RemoveContainer" containerID="1a62a0ca5587f8e8ff8d5726065b1727344c876317a507af4bdf77c62b1c3a37"
Sep 10 13:36:31.182596 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:36:31.182564    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a62a0ca5587f8e8ff8d5726065b1727344c876317a507af4bdf77c62b1c3a37\": container with ID starting with 1a62a0ca5587f8e8ff8d5726065b1727344c876317a507af4bdf77c62b1c3a37 not found: ID does not exist" containerID="1a62a0ca5587f8e8ff8d5726065b1727344c876317a507af4bdf77c62b1c3a37"
Sep 10 13:36:31.182642 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:31.182605    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a62a0ca5587f8e8ff8d5726065b1727344c876317a507af4bdf77c62b1c3a37"} err="failed to get container status \"1a62a0ca5587f8e8ff8d5726065b1727344c876317a507af4bdf77c62b1c3a37\": rpc error: code = NotFound desc = could not find container \"1a62a0ca5587f8e8ff8d5726065b1727344c876317a507af4bdf77c62b1c3a37\": container with ID starting with 1a62a0ca5587f8e8ff8d5726065b1727344c876317a507af4bdf77c62b1c3a37 not found: ID does not exist"
Sep 10 13:36:31.190415 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:31.190397    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"]
Sep 10 13:36:31.201297 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:31.201278    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-deba50-openshift-default-b5f6754fb-4gjkx"]
Sep 10 13:36:31.709629 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:31.709594    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" path="/var/lib/kubelet/pods/e232466f-ae92-4709-8a3b-8b1135ef8a9b/volumes"
Sep 10 13:36:44.095087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.095029    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj" podUID="94bed86a-e71e-4b8b-9db6-517ac1351dd8" containerName="main" containerID="cri-o://45e469762a6aec4c9f4686373eb9dc29a8e9a944bdaa0ee5c6ef192f8535d064" gracePeriod=45
Sep 10 13:36:44.233813 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.233704    2551 generic.go:356] "Generic (PLEG): container finished" podID="94bed86a-e71e-4b8b-9db6-517ac1351dd8" containerID="45e469762a6aec4c9f4686373eb9dc29a8e9a944bdaa0ee5c6ef192f8535d064" exitCode=0
Sep 10 13:36:44.233958 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.233772    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj" event={"ID":"94bed86a-e71e-4b8b-9db6-517ac1351dd8","Type":"ContainerDied","Data":"45e469762a6aec4c9f4686373eb9dc29a8e9a944bdaa0ee5c6ef192f8535d064"}
Sep 10 13:36:44.379047 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.379021    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:44.504687 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.504648    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8bbk\" (UniqueName: \"kubernetes.io/projected/94bed86a-e71e-4b8b-9db6-517ac1351dd8-kube-api-access-b8bbk\") pod \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") "
Sep 10 13:36:44.504858 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.504785    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-home\") pod \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") "
Sep 10 13:36:44.504858 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.504828    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94bed86a-e71e-4b8b-9db6-517ac1351dd8-tls-certs\") pod \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") "
Sep 10 13:36:44.504937 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.504862    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-dshm\") pod \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") "
Sep 10 13:36:44.504937 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.504890    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-model-cache\") pod \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") "
Sep 10 13:36:44.504937 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.504925    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-tmp-dir\") pod \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\" (UID: \"94bed86a-e71e-4b8b-9db6-517ac1351dd8\") "
Sep 10 13:36:44.505078 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.505051    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-home" (OuterVolumeSpecName: "home") pod "94bed86a-e71e-4b8b-9db6-517ac1351dd8" (UID: "94bed86a-e71e-4b8b-9db6-517ac1351dd8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:36:44.505234 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.505192    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-model-cache" (OuterVolumeSpecName: "model-cache") pod "94bed86a-e71e-4b8b-9db6-517ac1351dd8" (UID: "94bed86a-e71e-4b8b-9db6-517ac1351dd8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:36:44.505371 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.505243    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "94bed86a-e71e-4b8b-9db6-517ac1351dd8" (UID: "94bed86a-e71e-4b8b-9db6-517ac1351dd8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:36:44.505371 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.505271    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-home\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:44.506993 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.506968    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94bed86a-e71e-4b8b-9db6-517ac1351dd8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "94bed86a-e71e-4b8b-9db6-517ac1351dd8" (UID: "94bed86a-e71e-4b8b-9db6-517ac1351dd8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:36:44.506993 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.506986    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94bed86a-e71e-4b8b-9db6-517ac1351dd8-kube-api-access-b8bbk" (OuterVolumeSpecName: "kube-api-access-b8bbk") pod "94bed86a-e71e-4b8b-9db6-517ac1351dd8" (UID: "94bed86a-e71e-4b8b-9db6-517ac1351dd8"). InnerVolumeSpecName "kube-api-access-b8bbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:36:44.507115 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.507039    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-dshm" (OuterVolumeSpecName: "dshm") pod "94bed86a-e71e-4b8b-9db6-517ac1351dd8" (UID: "94bed86a-e71e-4b8b-9db6-517ac1351dd8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:36:44.606139 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.606102    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94bed86a-e71e-4b8b-9db6-517ac1351dd8-tls-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:44.606139 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.606141    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-dshm\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:44.606336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.606155    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-model-cache\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:44.606336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.606168    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94bed86a-e71e-4b8b-9db6-517ac1351dd8-tmp-dir\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:44.606336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:44.606181    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8bbk\" (UniqueName: \"kubernetes.io/projected/94bed86a-e71e-4b8b-9db6-517ac1351dd8-kube-api-access-b8bbk\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:45.024369 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.024330    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8"]
Sep 10 13:36:45.024786 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.024768    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" containerName="istio-proxy"
Sep 10 13:36:45.024883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.024788    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" containerName="istio-proxy"
Sep 10 13:36:45.024883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.024802    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94bed86a-e71e-4b8b-9db6-517ac1351dd8" containerName="main"
Sep 10 13:36:45.024883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.024810    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="94bed86a-e71e-4b8b-9db6-517ac1351dd8" containerName="main"
Sep 10 13:36:45.024883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.024826    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" containerName="istio-proxy"
Sep 10 13:36:45.024883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.024836    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" containerName="istio-proxy"
Sep 10 13:36:45.025133 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.024932    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" containerName="istio-proxy"
Sep 10 13:36:45.025133 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.024947    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e232466f-ae92-4709-8a3b-8b1135ef8a9b" containerName="istio-proxy"
Sep 10 13:36:45.025133 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.024957    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="94bed86a-e71e-4b8b-9db6-517ac1351dd8" containerName="main"
Sep 10 13:36:45.028371 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.028351    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8"
Sep 10 13:36:45.031109 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.031088    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:36:45.031109 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.031097    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:36:45.041697 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.041676    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8"]
Sep 10 13:36:45.109858 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.109818    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9af41fd3-4278-41a4-8535-e9d31f7a62c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8\" (UID: \"9af41fd3-4278-41a4-8535-e9d31f7a62c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8"
Sep 10 13:36:45.210590 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.210545    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9af41fd3-4278-41a4-8535-e9d31f7a62c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8\" (UID: \"9af41fd3-4278-41a4-8535-e9d31f7a62c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8"
Sep 10 13:36:45.211168 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.211147    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9af41fd3-4278-41a4-8535-e9d31f7a62c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8\" (UID: \"9af41fd3-4278-41a4-8535-e9d31f7a62c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8"
Sep 10 13:36:45.241366 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.241334    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"
Sep 10 13:36:45.241563 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.241365    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj" event={"ID":"94bed86a-e71e-4b8b-9db6-517ac1351dd8","Type":"ContainerDied","Data":"153a71ef5ab77dc523c9d9c244ea204e6affc8736a85749fbd38f6a5ef10face"}
Sep 10 13:36:45.241563 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.241420    2551 scope.go:117] "RemoveContainer" containerID="45e469762a6aec4c9f4686373eb9dc29a8e9a944bdaa0ee5c6ef192f8535d064"
Sep 10 13:36:45.262797 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.262773    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"]
Sep 10 13:36:45.266389 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.266368    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-3bafe380-kserve-7d797c5d69-dxnnj"]
Sep 10 13:36:45.338715 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.338622    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8"
Sep 10 13:36:45.370777 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.370743    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:36:45.618295 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.618254    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8"]
Sep 10 13:36:45.619412 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:36:45.619375    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9af41fd3_4278_41a4_8535_e9d31f7a62c7.slice/crio-a4c611a3536ca613bec2d9e5ccd1d5a749208890a5caf781561ed13b85a16c32 WatchSource:0}: Error finding container a4c611a3536ca613bec2d9e5ccd1d5a749208890a5caf781561ed13b85a16c32: Status 404 returned error can't find the container with id a4c611a3536ca613bec2d9e5ccd1d5a749208890a5caf781561ed13b85a16c32
Sep 10 13:36:45.710999 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:45.710957    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94bed86a-e71e-4b8b-9db6-517ac1351dd8" path="/var/lib/kubelet/pods/94bed86a-e71e-4b8b-9db6-517ac1351dd8/volumes"
Sep 10 13:36:46.249704 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:46.249557    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8" event={"ID":"9af41fd3-4278-41a4-8535-e9d31f7a62c7","Type":"ContainerStarted","Data":"201bb7e4c5baf74658d337b2611123586775ecbdb9d98f8c886c2f3ffa756193"}
Sep 10 13:36:46.249704 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:46.249682    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8" event={"ID":"9af41fd3-4278-41a4-8535-e9d31f7a62c7","Type":"ContainerStarted","Data":"a4c611a3536ca613bec2d9e5ccd1d5a749208890a5caf781561ed13b85a16c32"}
Sep 10 13:36:46.269991 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:46.269938    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8" podStartSLOduration=2.269919886 podStartE2EDuration="2.269919886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:46.265794872 +0000 UTC m=+1117.084833274" watchObservedRunningTime="2026-09-10 13:36:46.269919886 +0000 UTC m=+1117.088958289"
Sep 10 13:36:47.258337 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:47.258303    2551 generic.go:356] "Generic (PLEG): container finished" podID="9af41fd3-4278-41a4-8535-e9d31f7a62c7" containerID="201bb7e4c5baf74658d337b2611123586775ecbdb9d98f8c886c2f3ffa756193" exitCode=0
Sep 10 13:36:47.258720 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:47.258373    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8" event={"ID":"9af41fd3-4278-41a4-8535-e9d31f7a62c7","Type":"ContainerDied","Data":"201bb7e4c5baf74658d337b2611123586775ecbdb9d98f8c886c2f3ffa756193"}
Sep 10 13:36:48.264126 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:48.264094    2551 generic.go:356] "Generic (PLEG): container finished" podID="8612a427-bc5b-4ecd-9ba1-2c9399912173" containerID="e90b849bdb8642547685321ca1798d01d02f07570802ca55136fc92cb14b51b1" exitCode=0
Sep 10 13:36:48.264542 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:48.264168    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw" event={"ID":"8612a427-bc5b-4ecd-9ba1-2c9399912173","Type":"ContainerDied","Data":"e90b849bdb8642547685321ca1798d01d02f07570802ca55136fc92cb14b51b1"}
Sep 10 13:36:48.264947 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:48.264664    2551 scope.go:117] "RemoveContainer" containerID="e90b849bdb8642547685321ca1798d01d02f07570802ca55136fc92cb14b51b1"
Sep 10 13:36:48.265745 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:48.265709    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:36:48.265872 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:48.265785    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:36:48.265872 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:48.265816    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:36:48.409381 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:48.409355    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8"
Sep 10 13:36:48.442264 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:48.442233    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9af41fd3-4278-41a4-8535-e9d31f7a62c7-maas-api-service-ca\") pod \"9af41fd3-4278-41a4-8535-e9d31f7a62c7\" (UID: \"9af41fd3-4278-41a4-8535-e9d31f7a62c7\") "
Sep 10 13:36:48.442544 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:48.442523    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9af41fd3-4278-41a4-8535-e9d31f7a62c7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9af41fd3-4278-41a4-8535-e9d31f7a62c7" (UID: "9af41fd3-4278-41a4-8535-e9d31f7a62c7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:36:48.543830 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:48.543754    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9af41fd3-4278-41a4-8535-e9d31f7a62c7-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:36:49.273412 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:49.273370    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw" event={"ID":"8612a427-bc5b-4ecd-9ba1-2c9399912173","Type":"ContainerStarted","Data":"7fa19316e7a2ee7598a5341a94b222c1b74d206e86b6f2d378560e27f4eb436e"}
Sep 10 13:36:49.274823 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:49.274801    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8"
Sep 10 13:36:49.274954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:49.274839    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8" event={"ID":"9af41fd3-4278-41a4-8535-e9d31f7a62c7","Type":"ContainerDied","Data":"a4c611a3536ca613bec2d9e5ccd1d5a749208890a5caf781561ed13b85a16c32"}
Sep 10 13:36:49.274954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:49.274877    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4c611a3536ca613bec2d9e5ccd1d5a749208890a5caf781561ed13b85a16c32"
Sep 10 13:36:50.137647 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:50.137612    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:50.137847 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:50.137830    2551 patch_prober.go:28] interesting pod/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 10 13:36:50.137896 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:50.137873    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw" podUID="8612a427-bc5b-4ecd-9ba1-2c9399912173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 10 13:36:51.137503 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:51.137470    2551 patch_prober.go:28] interesting pod/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 10 13:36:51.137899 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:51.137524    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw" podUID="8612a427-bc5b-4ecd-9ba1-2c9399912173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 10 13:36:51.559923 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:51.559886    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8"]
Sep 10 13:36:51.563237 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:51.563213    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-95ae6288-6g5c8"]
Sep 10 13:36:51.709596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:51.709540    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9af41fd3-4278-41a4-8535-e9d31f7a62c7" path="/var/lib/kubelet/pods/9af41fd3-4278-41a4-8535-e9d31f7a62c7/volumes"
Sep 10 13:36:52.137720 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:52.137678    2551 patch_prober.go:28] interesting pod/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 10 13:36:52.138093 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:52.137738    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw" podUID="8612a427-bc5b-4ecd-9ba1-2c9399912173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 10 13:36:53.137320 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:53.137283    2551 patch_prober.go:28] interesting pod/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 10 13:36:53.137501 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:53.137338    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw" podUID="8612a427-bc5b-4ecd-9ba1-2c9399912173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 10 13:36:54.136897 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:54.136863    2551 patch_prober.go:28] interesting pod/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 10 13:36:54.137264 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:54.136916    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw" podUID="8612a427-bc5b-4ecd-9ba1-2c9399912173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 10 13:36:55.140889 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:55.140859    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:55.141262 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:55.141125    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:55.141801 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:55.141783    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:36:56.871519 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:36:56.871490    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"]
Sep 10 13:37:03.480912 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:03.480867    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:37:11.890623 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:11.890555    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh" podUID="0a4e2303-9505-473f-8ae1-8d0e68355ebb" containerName="main" containerID="cri-o://c1a38d2d9fef3e39ca49e12b7cc9acae133620081be9abe7dab396cdb5fe7846" gracePeriod=45
Sep 10 13:37:12.139869 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.139845    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:37:12.257276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.257248    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-model-cache\") pod \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") "
Sep 10 13:37:12.257430 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.257285    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-tmp-dir\") pod \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") "
Sep 10 13:37:12.257430 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.257307    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a4e2303-9505-473f-8ae1-8d0e68355ebb-tls-certs\") pod \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") "
Sep 10 13:37:12.257430 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.257353    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtbgz\" (UniqueName: \"kubernetes.io/projected/0a4e2303-9505-473f-8ae1-8d0e68355ebb-kube-api-access-wtbgz\") pod \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") "
Sep 10 13:37:12.257430 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.257386    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-dshm\") pod \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") "
Sep 10 13:37:12.257678 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.257446    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-home\") pod \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\" (UID: \"0a4e2303-9505-473f-8ae1-8d0e68355ebb\") "
Sep 10 13:37:12.257678 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.257501    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-model-cache" (OuterVolumeSpecName: "model-cache") pod "0a4e2303-9505-473f-8ae1-8d0e68355ebb" (UID: "0a4e2303-9505-473f-8ae1-8d0e68355ebb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:37:12.257678 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.257542    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0a4e2303-9505-473f-8ae1-8d0e68355ebb" (UID: "0a4e2303-9505-473f-8ae1-8d0e68355ebb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:37:12.257840 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.257759    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-home" (OuterVolumeSpecName: "home") pod "0a4e2303-9505-473f-8ae1-8d0e68355ebb" (UID: "0a4e2303-9505-473f-8ae1-8d0e68355ebb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:37:12.257840 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.257832    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-home\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:12.257929 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.257853    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-model-cache\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:12.257929 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.257869    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-tmp-dir\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:12.259501 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.259472    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a4e2303-9505-473f-8ae1-8d0e68355ebb-kube-api-access-wtbgz" (OuterVolumeSpecName: "kube-api-access-wtbgz") pod "0a4e2303-9505-473f-8ae1-8d0e68355ebb" (UID: "0a4e2303-9505-473f-8ae1-8d0e68355ebb"). InnerVolumeSpecName "kube-api-access-wtbgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:37:12.259501 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.259493    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a4e2303-9505-473f-8ae1-8d0e68355ebb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0a4e2303-9505-473f-8ae1-8d0e68355ebb" (UID: "0a4e2303-9505-473f-8ae1-8d0e68355ebb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:37:12.259668 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.259499    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-dshm" (OuterVolumeSpecName: "dshm") pod "0a4e2303-9505-473f-8ae1-8d0e68355ebb" (UID: "0a4e2303-9505-473f-8ae1-8d0e68355ebb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:37:12.358891 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.358847    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a4e2303-9505-473f-8ae1-8d0e68355ebb-tls-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:12.358891 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.358887    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtbgz\" (UniqueName: \"kubernetes.io/projected/0a4e2303-9505-473f-8ae1-8d0e68355ebb-kube-api-access-wtbgz\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:12.358891 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.358896    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a4e2303-9505-473f-8ae1-8d0e68355ebb-dshm\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:12.388063 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.388032    2551 generic.go:356] "Generic (PLEG): container finished" podID="0a4e2303-9505-473f-8ae1-8d0e68355ebb" containerID="c1a38d2d9fef3e39ca49e12b7cc9acae133620081be9abe7dab396cdb5fe7846" exitCode=0
Sep 10 13:37:12.388171 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.388076    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"
Sep 10 13:37:12.388171 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.388093    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh" event={"ID":"0a4e2303-9505-473f-8ae1-8d0e68355ebb","Type":"ContainerDied","Data":"c1a38d2d9fef3e39ca49e12b7cc9acae133620081be9abe7dab396cdb5fe7846"}
Sep 10 13:37:12.388171 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.388123    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh" event={"ID":"0a4e2303-9505-473f-8ae1-8d0e68355ebb","Type":"ContainerDied","Data":"daadef57d34a4765ffcc1ed1b0ec963fbbdf6ec2df6c0ac1926d2d73a3b64205"}
Sep 10 13:37:12.388171 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.388139    2551 scope.go:117] "RemoveContainer" containerID="c1a38d2d9fef3e39ca49e12b7cc9acae133620081be9abe7dab396cdb5fe7846"
Sep 10 13:37:12.396779 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.396761    2551 scope.go:117] "RemoveContainer" containerID="c1a38d2d9fef3e39ca49e12b7cc9acae133620081be9abe7dab396cdb5fe7846"
Sep 10 13:37:12.397043 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:37:12.397023    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1a38d2d9fef3e39ca49e12b7cc9acae133620081be9abe7dab396cdb5fe7846\": container with ID starting with c1a38d2d9fef3e39ca49e12b7cc9acae133620081be9abe7dab396cdb5fe7846 not found: ID does not exist" containerID="c1a38d2d9fef3e39ca49e12b7cc9acae133620081be9abe7dab396cdb5fe7846"
Sep 10 13:37:12.397098 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.397050    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1a38d2d9fef3e39ca49e12b7cc9acae133620081be9abe7dab396cdb5fe7846"} err="failed to get container status \"c1a38d2d9fef3e39ca49e12b7cc9acae133620081be9abe7dab396cdb5fe7846\": rpc error: code = NotFound desc = could not find container \"c1a38d2d9fef3e39ca49e12b7cc9acae133620081be9abe7dab396cdb5fe7846\": container with ID starting with c1a38d2d9fef3e39ca49e12b7cc9acae133620081be9abe7dab396cdb5fe7846 not found: ID does not exist"
Sep 10 13:37:12.416151 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.416129    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"]
Sep 10 13:37:12.421275 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:12.421246    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-3bafe380-kserve-6b66846775-sr6qh"]
Sep 10 13:37:13.711413 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:13.711377    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a4e2303-9505-473f-8ae1-8d0e68355ebb" path="/var/lib/kubelet/pods/0a4e2303-9505-473f-8ae1-8d0e68355ebb/volumes"
Sep 10 13:37:17.780717 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.780686    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"]
Sep 10 13:37:17.781163 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.781025    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a4e2303-9505-473f-8ae1-8d0e68355ebb" containerName="main"
Sep 10 13:37:17.781163 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.781036    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a4e2303-9505-473f-8ae1-8d0e68355ebb" containerName="main"
Sep 10 13:37:17.781163 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.781045    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9af41fd3-4278-41a4-8535-e9d31f7a62c7" containerName="revoke-keys"
Sep 10 13:37:17.781163 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.781051    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9af41fd3-4278-41a4-8535-e9d31f7a62c7" containerName="revoke-keys"
Sep 10 13:37:17.781163 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.781116    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a4e2303-9505-473f-8ae1-8d0e68355ebb" containerName="main"
Sep 10 13:37:17.781163 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.781125    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9af41fd3-4278-41a4-8535-e9d31f7a62c7" containerName="revoke-keys"
Sep 10 13:37:17.785255 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.785229    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:17.788946 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.788915    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-3c32b1\"/\"default-dockercfg-btnz7\""
Sep 10 13:37:17.789076 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.788950    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-3c32b1\"/\"explicit-model-3c32b1-kserve-self-signed-certs\""
Sep 10 13:37:17.789076 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.788963    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-3c32b1\"/\"kube-root-ca.crt\""
Sep 10 13:37:17.789076 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.788971    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-3c32b1\"/\"openshift-service-ca.crt\""
Sep 10 13:37:17.796350 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.796331    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"]
Sep 10 13:37:17.911468 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.911435    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-dshm\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:17.911468 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.911466    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1ede543-04c8-4c3d-a276-92c260c2b430-tls-certs\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:17.911738 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.911495    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-home\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:17.911738 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.911652    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-tmp-dir\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:17.911738 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.911699    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbmj4\" (UniqueName: \"kubernetes.io/projected/f1ede543-04c8-4c3d-a276-92c260c2b430-kube-api-access-vbmj4\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:17.911879 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:17.911753    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-model-cache\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:18.012689 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.012651    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-dshm\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:18.012844 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.012696    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1ede543-04c8-4c3d-a276-92c260c2b430-tls-certs\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:18.012844 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.012747    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-home\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:18.012844 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.012826    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-tmp-dir\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:18.013038 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.012864    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbmj4\" (UniqueName: \"kubernetes.io/projected/f1ede543-04c8-4c3d-a276-92c260c2b430-kube-api-access-vbmj4\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:18.013038 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.013001    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-model-cache\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:18.013194 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.013175    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-home\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:18.013247 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.013233    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-tmp-dir\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:18.013326 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.013308    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-model-cache\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:18.014965 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.014944    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-dshm\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:18.015212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.015193    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1ede543-04c8-4c3d-a276-92c260c2b430-tls-certs\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:18.028723 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.028694    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbmj4\" (UniqueName: \"kubernetes.io/projected/f1ede543-04c8-4c3d-a276-92c260c2b430-kube-api-access-vbmj4\") pod \"explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") " pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:18.094403 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.094345    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:18.368991 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.368923    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"]
Sep 10 13:37:18.420188 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.420164    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8" event={"ID":"f1ede543-04c8-4c3d-a276-92c260c2b430","Type":"ContainerStarted","Data":"bcad360af6b974b8f94aa8f254b8a1da45fe4cc578e2b90efcfb0b19efc6deca"}
Sep 10 13:37:18.977216 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:18.977173    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:37:19.426945 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:19.426863    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8" event={"ID":"f1ede543-04c8-4c3d-a276-92c260c2b430","Type":"ContainerStarted","Data":"bc20c8e8f84ea8480e3fa1e2c0e9776eaac40ee62e18f0274b2fb254a908104f"}
Sep 10 13:37:19.454493 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:19.454441    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8" podStartSLOduration=2.454429305 podStartE2EDuration="2.454429305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:37:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:37:19.453358475 +0000 UTC m=+1150.272396879" watchObservedRunningTime="2026-09-10 13:37:19.454429305 +0000 UTC m=+1150.273467704"
Sep 10 13:37:21.303398 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:21.303365    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs"]
Sep 10 13:37:21.306912 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:21.306896    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs"
Sep 10 13:37:21.312378 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:21.312355    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:37:21.312523 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:21.312404    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:37:21.318865 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:21.318843    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs"]
Sep 10 13:37:21.444469 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:21.444444    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e27ee76-818e-4270-95f4-077da9007184-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs\" (UID: \"1e27ee76-818e-4270-95f4-077da9007184\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs"
Sep 10 13:37:21.545736 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:21.545700    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e27ee76-818e-4270-95f4-077da9007184-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs\" (UID: \"1e27ee76-818e-4270-95f4-077da9007184\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs"
Sep 10 13:37:21.546349 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:21.546321    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e27ee76-818e-4270-95f4-077da9007184-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs\" (UID: \"1e27ee76-818e-4270-95f4-077da9007184\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs"
Sep 10 13:37:21.617842 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:21.617779    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs"
Sep 10 13:37:21.889881 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:21.889853    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs"]
Sep 10 13:37:21.891332 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:37:21.891307    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e27ee76_818e_4270_95f4_077da9007184.slice/crio-2b4538725c553506306c749e349739255986c01ebd0ba764c0dd7643fb8ea882 WatchSource:0}: Error finding container 2b4538725c553506306c749e349739255986c01ebd0ba764c0dd7643fb8ea882: Status 404 returned error can't find the container with id 2b4538725c553506306c749e349739255986c01ebd0ba764c0dd7643fb8ea882
Sep 10 13:37:22.443498 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.443465    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs" event={"ID":"1e27ee76-818e-4270-95f4-077da9007184","Type":"ContainerStarted","Data":"5d232a6b6e6d23da43ac2ca1b14cc4017b6655a476815033321ab514491d742b"}
Sep 10 13:37:22.443962 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.443515    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs" event={"ID":"1e27ee76-818e-4270-95f4-077da9007184","Type":"ContainerStarted","Data":"2b4538725c553506306c749e349739255986c01ebd0ba764c0dd7643fb8ea882"}
Sep 10 13:37:22.462748 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.462707    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs" podStartSLOduration=1.462693724 podStartE2EDuration="1.462693724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:37:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:37:22.459886159 +0000 UTC m=+1153.278924560" watchObservedRunningTime="2026-09-10 13:37:22.462693724 +0000 UTC m=+1153.281732123"
Sep 10 13:37:22.681818 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.681786    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"]
Sep 10 13:37:22.684981 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.684963    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.688949 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.688916    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d6159c\"/\"default-dockercfg-bgx8h\""
Sep 10 13:37:22.688949 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.688938    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d6159c\"/\"openshift-service-ca.crt\""
Sep 10 13:37:22.689236 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.689220    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d6159c\"/\"kube-root-ca.crt\""
Sep 10 13:37:22.689977 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.689962    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d6159c\"/\"test-model-d6159c-kserve-self-signed-certs\""
Sep 10 13:37:22.703015 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.702961    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"]
Sep 10 13:37:22.856950 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.856918    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-home\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.857139 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.856982    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-model-cache\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.857139 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.857109    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-dshm\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.857265 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.857156    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-tmp-dir\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.857265 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.857194    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcdb5\" (UniqueName: \"kubernetes.io/projected/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-kube-api-access-fcdb5\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.857376 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.857319    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-tls-certs\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.958204 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.958095    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-tmp-dir\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.958204 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.958143    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcdb5\" (UniqueName: \"kubernetes.io/projected/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-kube-api-access-fcdb5\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.958204 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.958169    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-tls-certs\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.958204 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.958210    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-home\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.958639 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.958238    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-model-cache\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.958639 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.958306    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-dshm\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.958639 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.958508    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-tmp-dir\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.958639 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.958559    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-home\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.958879 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.958697    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-model-cache\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.960447 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.960420    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-dshm\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.960659 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.960640    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-tls-certs\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.968767 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.968735    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcdb5\" (UniqueName: \"kubernetes.io/projected/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-kube-api-access-fcdb5\") pod \"test-model-d6159c-kserve-7c454885b5-v4nx4\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") " pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:22.994667 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:22.994646    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:23.266676 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:23.266652    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"]
Sep 10 13:37:23.268417 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:37:23.268396    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39e46455_71e4_4d3f_9d9b_04b2b6006c9e.slice/crio-a7972828faffbdbbfe26526694d5be99f3375944bae037fe917ed21d405f2e2b WatchSource:0}: Error finding container a7972828faffbdbbfe26526694d5be99f3375944bae037fe917ed21d405f2e2b: Status 404 returned error can't find the container with id a7972828faffbdbbfe26526694d5be99f3375944bae037fe917ed21d405f2e2b
Sep 10 13:37:23.449839 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:23.449804    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4" event={"ID":"39e46455-71e4-4d3f-9d9b-04b2b6006c9e","Type":"ContainerStarted","Data":"84588c5fa58f77d85af4d2db6aa136ad8607a805b946f9efc97f4ddc902b9a1e"}
Sep 10 13:37:23.450203 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:23.449851    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4" event={"ID":"39e46455-71e4-4d3f-9d9b-04b2b6006c9e","Type":"ContainerStarted","Data":"a7972828faffbdbbfe26526694d5be99f3375944bae037fe917ed21d405f2e2b"}
Sep 10 13:37:23.451364 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:23.451341    2551 generic.go:356] "Generic (PLEG): container finished" podID="1e27ee76-818e-4270-95f4-077da9007184" containerID="5d232a6b6e6d23da43ac2ca1b14cc4017b6655a476815033321ab514491d742b" exitCode=0
Sep 10 13:37:23.451474 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:23.451420    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs" event={"ID":"1e27ee76-818e-4270-95f4-077da9007184","Type":"ContainerDied","Data":"5d232a6b6e6d23da43ac2ca1b14cc4017b6655a476815033321ab514491d742b"}
Sep 10 13:37:23.477554 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:23.477508    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4" podStartSLOduration=1.477495543 podStartE2EDuration="1.477495543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:37:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:37:23.476470343 +0000 UTC m=+1154.295508742" watchObservedRunningTime="2026-09-10 13:37:23.477495543 +0000 UTC m=+1154.296533942"
Sep 10 13:37:24.581075 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:24.581046    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs"
Sep 10 13:37:24.671357 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:24.671328    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e27ee76-818e-4270-95f4-077da9007184-maas-api-service-ca\") pod \"1e27ee76-818e-4270-95f4-077da9007184\" (UID: \"1e27ee76-818e-4270-95f4-077da9007184\") "
Sep 10 13:37:24.671661 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:24.671641    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e27ee76-818e-4270-95f4-077da9007184-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1e27ee76-818e-4270-95f4-077da9007184" (UID: "1e27ee76-818e-4270-95f4-077da9007184"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:37:24.773001 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:24.772976    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e27ee76-818e-4270-95f4-077da9007184-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:25.463982 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:25.463953    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs"
Sep 10 13:37:25.464154 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:25.463957    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs" event={"ID":"1e27ee76-818e-4270-95f4-077da9007184","Type":"ContainerDied","Data":"2b4538725c553506306c749e349739255986c01ebd0ba764c0dd7643fb8ea882"}
Sep 10 13:37:25.464154 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:25.464078    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b4538725c553506306c749e349739255986c01ebd0ba764c0dd7643fb8ea882"
Sep 10 13:37:26.180138 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:26.180107    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"]
Sep 10 13:37:26.294708 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:26.294666    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f"]
Sep 10 13:37:26.295038 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:26.295023    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e27ee76-818e-4270-95f4-077da9007184" containerName="revoke-keys"
Sep 10 13:37:26.295038 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:26.295038    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e27ee76-818e-4270-95f4-077da9007184" containerName="revoke-keys"
Sep 10 13:37:26.295174 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:26.295111    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e27ee76-818e-4270-95f4-077da9007184" containerName="revoke-keys"
Sep 10 13:37:26.298090 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:26.298067    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f"
Sep 10 13:37:26.303362 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:26.303344    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:37:26.303467 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:26.303344    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:37:26.310839 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:26.310818    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f"]
Sep 10 13:37:26.387376 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:26.387336    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89b0fe9c-7c07-4be0-a391-6bb97b49d0c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f\" (UID: \"89b0fe9c-7c07-4be0-a391-6bb97b49d0c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f"
Sep 10 13:37:26.488479 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:26.488451    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89b0fe9c-7c07-4be0-a391-6bb97b49d0c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f\" (UID: \"89b0fe9c-7c07-4be0-a391-6bb97b49d0c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f"
Sep 10 13:37:26.489124 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:26.489103    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89b0fe9c-7c07-4be0-a391-6bb97b49d0c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f\" (UID: \"89b0fe9c-7c07-4be0-a391-6bb97b49d0c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f"
Sep 10 13:37:26.610901 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:26.610874    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f"
Sep 10 13:37:26.883409 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:26.883386    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f"]
Sep 10 13:37:26.884978 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:37:26.884952    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89b0fe9c_7c07_4be0_a391_6bb97b49d0c9.slice/crio-e992262eeab3f595727e63446e0ef111d16eb5069092ab4a54234cd18f078b5e WatchSource:0}: Error finding container e992262eeab3f595727e63446e0ef111d16eb5069092ab4a54234cd18f078b5e: Status 404 returned error can't find the container with id e992262eeab3f595727e63446e0ef111d16eb5069092ab4a54234cd18f078b5e
Sep 10 13:37:27.476228 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:27.476187    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f" event={"ID":"89b0fe9c-7c07-4be0-a391-6bb97b49d0c9","Type":"ContainerStarted","Data":"9fa4c75a41ac30622f59c54238d35da15d7cc7d3efc4abbc8c02b9d263410574"}
Sep 10 13:37:27.476685 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:27.476270    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f" event={"ID":"89b0fe9c-7c07-4be0-a391-6bb97b49d0c9","Type":"ContainerStarted","Data":"e992262eeab3f595727e63446e0ef111d16eb5069092ab4a54234cd18f078b5e"}
Sep 10 13:37:27.507084 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:27.507029    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f" podStartSLOduration=1.507010174 podStartE2EDuration="1.507010174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:37:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:37:27.506039682 +0000 UTC m=+1158.325078082" watchObservedRunningTime="2026-09-10 13:37:27.507010174 +0000 UTC m=+1158.326048575"
Sep 10 13:37:28.095398 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:28.095368    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:28.207701 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:28.207674    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs"]
Sep 10 13:37:28.211852 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:28.211820    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2e7a3258-zf6bs"]
Sep 10 13:37:28.483138 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:28.483101    2551 generic.go:356] "Generic (PLEG): container finished" podID="89b0fe9c-7c07-4be0-a391-6bb97b49d0c9" containerID="9fa4c75a41ac30622f59c54238d35da15d7cc7d3efc4abbc8c02b9d263410574" exitCode=0
Sep 10 13:37:28.483511 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:28.483158    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f" event={"ID":"89b0fe9c-7c07-4be0-a391-6bb97b49d0c9","Type":"ContainerDied","Data":"9fa4c75a41ac30622f59c54238d35da15d7cc7d3efc4abbc8c02b9d263410574"}
Sep 10 13:37:29.673260 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:29.673232    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f"
Sep 10 13:37:29.715641 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:29.715606    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e27ee76-818e-4270-95f4-077da9007184" path="/var/lib/kubelet/pods/1e27ee76-818e-4270-95f4-077da9007184/volumes"
Sep 10 13:37:29.725588 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:29.725538    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89b0fe9c-7c07-4be0-a391-6bb97b49d0c9-maas-api-service-ca\") pod \"89b0fe9c-7c07-4be0-a391-6bb97b49d0c9\" (UID: \"89b0fe9c-7c07-4be0-a391-6bb97b49d0c9\") "
Sep 10 13:37:29.726123 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:29.726087    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89b0fe9c-7c07-4be0-a391-6bb97b49d0c9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "89b0fe9c-7c07-4be0-a391-6bb97b49d0c9" (UID: "89b0fe9c-7c07-4be0-a391-6bb97b49d0c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:37:29.826378 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:29.826309    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89b0fe9c-7c07-4be0-a391-6bb97b49d0c9-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:30.496046 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:30.496012    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f" event={"ID":"89b0fe9c-7c07-4be0-a391-6bb97b49d0c9","Type":"ContainerDied","Data":"e992262eeab3f595727e63446e0ef111d16eb5069092ab4a54234cd18f078b5e"}
Sep 10 13:37:30.496046 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:30.496035    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f"
Sep 10 13:37:30.496252 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:30.496062    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e992262eeab3f595727e63446e0ef111d16eb5069092ab4a54234cd18f078b5e"
Sep 10 13:37:32.995400 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:32.995322    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:32.995400 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:32.995373    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:33.007341 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:33.007318    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:33.520989 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:33.520960    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:37:37.446635 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:37.446598    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f"]
Sep 10 13:37:37.448486 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:37.448460    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3c32b1-wzq8f"]
Sep 10 13:37:37.709929 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:37.709850    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89b0fe9c-7c07-4be0-a391-6bb97b49d0c9" path="/var/lib/kubelet/pods/89b0fe9c-7c07-4be0-a391-6bb97b49d0c9/volumes"
Sep 10 13:37:38.093346 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:38.093308    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:37:41.198315 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.198253    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8" podUID="f1ede543-04c8-4c3d-a276-92c260c2b430" containerName="main" containerID="cri-o://bc20c8e8f84ea8480e3fa1e2c0e9776eaac40ee62e18f0274b2fb254a908104f" gracePeriod=45
Sep 10 13:37:41.437265 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.437239    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:41.531365 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.531336    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-home\") pod \"f1ede543-04c8-4c3d-a276-92c260c2b430\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") "
Sep 10 13:37:41.531513 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.531390    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-dshm\") pod \"f1ede543-04c8-4c3d-a276-92c260c2b430\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") "
Sep 10 13:37:41.531513 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.531480    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-tmp-dir\") pod \"f1ede543-04c8-4c3d-a276-92c260c2b430\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") "
Sep 10 13:37:41.531668 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.531511    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbmj4\" (UniqueName: \"kubernetes.io/projected/f1ede543-04c8-4c3d-a276-92c260c2b430-kube-api-access-vbmj4\") pod \"f1ede543-04c8-4c3d-a276-92c260c2b430\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") "
Sep 10 13:37:41.531668 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.531563    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-home" (OuterVolumeSpecName: "home") pod "f1ede543-04c8-4c3d-a276-92c260c2b430" (UID: "f1ede543-04c8-4c3d-a276-92c260c2b430"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:37:41.531668 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.531594    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-model-cache\") pod \"f1ede543-04c8-4c3d-a276-92c260c2b430\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") "
Sep 10 13:37:41.531830 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.531673    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1ede543-04c8-4c3d-a276-92c260c2b430-tls-certs\") pod \"f1ede543-04c8-4c3d-a276-92c260c2b430\" (UID: \"f1ede543-04c8-4c3d-a276-92c260c2b430\") "
Sep 10 13:37:41.531830 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.531721    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f1ede543-04c8-4c3d-a276-92c260c2b430" (UID: "f1ede543-04c8-4c3d-a276-92c260c2b430"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:37:41.531830 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.531801    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-model-cache" (OuterVolumeSpecName: "model-cache") pod "f1ede543-04c8-4c3d-a276-92c260c2b430" (UID: "f1ede543-04c8-4c3d-a276-92c260c2b430"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:37:41.532009 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.531994    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-model-cache\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:41.532059 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.532014    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-home\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:41.532059 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.532027    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-tmp-dir\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:41.533511 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.533486    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-dshm" (OuterVolumeSpecName: "dshm") pod "f1ede543-04c8-4c3d-a276-92c260c2b430" (UID: "f1ede543-04c8-4c3d-a276-92c260c2b430"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:37:41.533617 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.533553    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1ede543-04c8-4c3d-a276-92c260c2b430-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f1ede543-04c8-4c3d-a276-92c260c2b430" (UID: "f1ede543-04c8-4c3d-a276-92c260c2b430"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:37:41.533617 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.533558    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1ede543-04c8-4c3d-a276-92c260c2b430-kube-api-access-vbmj4" (OuterVolumeSpecName: "kube-api-access-vbmj4") pod "f1ede543-04c8-4c3d-a276-92c260c2b430" (UID: "f1ede543-04c8-4c3d-a276-92c260c2b430"). InnerVolumeSpecName "kube-api-access-vbmj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:37:41.550913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.550890    2551 generic.go:356] "Generic (PLEG): container finished" podID="f1ede543-04c8-4c3d-a276-92c260c2b430" containerID="bc20c8e8f84ea8480e3fa1e2c0e9776eaac40ee62e18f0274b2fb254a908104f" exitCode=0
Sep 10 13:37:41.550999 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.550929    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"
Sep 10 13:37:41.550999 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.550969    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8" event={"ID":"f1ede543-04c8-4c3d-a276-92c260c2b430","Type":"ContainerDied","Data":"bc20c8e8f84ea8480e3fa1e2c0e9776eaac40ee62e18f0274b2fb254a908104f"}
Sep 10 13:37:41.551088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.551008    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8" event={"ID":"f1ede543-04c8-4c3d-a276-92c260c2b430","Type":"ContainerDied","Data":"bcad360af6b974b8f94aa8f254b8a1da45fe4cc578e2b90efcfb0b19efc6deca"}
Sep 10 13:37:41.551088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.551029    2551 scope.go:117] "RemoveContainer" containerID="bc20c8e8f84ea8480e3fa1e2c0e9776eaac40ee62e18f0274b2fb254a908104f"
Sep 10 13:37:41.559111 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.559096    2551 scope.go:117] "RemoveContainer" containerID="bc20c8e8f84ea8480e3fa1e2c0e9776eaac40ee62e18f0274b2fb254a908104f"
Sep 10 13:37:41.559346 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:37:41.559328    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc20c8e8f84ea8480e3fa1e2c0e9776eaac40ee62e18f0274b2fb254a908104f\": container with ID starting with bc20c8e8f84ea8480e3fa1e2c0e9776eaac40ee62e18f0274b2fb254a908104f not found: ID does not exist" containerID="bc20c8e8f84ea8480e3fa1e2c0e9776eaac40ee62e18f0274b2fb254a908104f"
Sep 10 13:37:41.559392 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.559355    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc20c8e8f84ea8480e3fa1e2c0e9776eaac40ee62e18f0274b2fb254a908104f"} err="failed to get container status \"bc20c8e8f84ea8480e3fa1e2c0e9776eaac40ee62e18f0274b2fb254a908104f\": rpc error: code = NotFound desc = could not find container \"bc20c8e8f84ea8480e3fa1e2c0e9776eaac40ee62e18f0274b2fb254a908104f\": container with ID starting with bc20c8e8f84ea8480e3fa1e2c0e9776eaac40ee62e18f0274b2fb254a908104f not found: ID does not exist"
Sep 10 13:37:41.574762 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.574740    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"]
Sep 10 13:37:41.580360 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.580313    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-3c32b1/explicit-model-3c32b1-kserve-5fd6869bdf-8bbf8"]
Sep 10 13:37:41.632438 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.632419    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1ede543-04c8-4c3d-a276-92c260c2b430-dshm\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:41.632438 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.632438    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbmj4\" (UniqueName: \"kubernetes.io/projected/f1ede543-04c8-4c3d-a276-92c260c2b430-kube-api-access-vbmj4\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:41.632557 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.632448    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1ede543-04c8-4c3d-a276-92c260c2b430-tls-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:41.709753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:41.709729    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1ede543-04c8-4c3d-a276-92c260c2b430" path="/var/lib/kubelet/pods/f1ede543-04c8-4c3d-a276-92c260c2b430/volumes"
Sep 10 13:37:44.691634 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:44.691600    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"]
Sep 10 13:37:44.692001 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:44.691860    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw" podUID="8612a427-bc5b-4ecd-9ba1-2c9399912173" containerName="istio-proxy" containerID="cri-o://7fa19316e7a2ee7598a5341a94b222c1b74d206e86b6f2d378560e27f4eb436e" gracePeriod=30
Sep 10 13:37:49.949307 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:49.949278    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:37:50.002208 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002181    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8612a427-bc5b-4ecd-9ba1-2c9399912173-istiod-ca-cert\") pod \"8612a427-bc5b-4ecd-9ba1-2c9399912173\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") "
Sep 10 13:37:50.002208 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002212    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-workload-socket\") pod \"8612a427-bc5b-4ecd-9ba1-2c9399912173\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") "
Sep 10 13:37:50.002436 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002249    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-data\") pod \"8612a427-bc5b-4ecd-9ba1-2c9399912173\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") "
Sep 10 13:37:50.002436 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002293    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-credential-socket\") pod \"8612a427-bc5b-4ecd-9ba1-2c9399912173\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") "
Sep 10 13:37:50.002436 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002319    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-token\") pod \"8612a427-bc5b-4ecd-9ba1-2c9399912173\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") "
Sep 10 13:37:50.002436 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002373    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-podinfo\") pod \"8612a427-bc5b-4ecd-9ba1-2c9399912173\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") "
Sep 10 13:37:50.002436 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002405    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-envoy\") pod \"8612a427-bc5b-4ecd-9ba1-2c9399912173\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") "
Sep 10 13:37:50.002436 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002428    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-workload-certs\") pod \"8612a427-bc5b-4ecd-9ba1-2c9399912173\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") "
Sep 10 13:37:50.002805 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002455    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h75wp\" (UniqueName: \"kubernetes.io/projected/8612a427-bc5b-4ecd-9ba1-2c9399912173-kube-api-access-h75wp\") pod \"8612a427-bc5b-4ecd-9ba1-2c9399912173\" (UID: \"8612a427-bc5b-4ecd-9ba1-2c9399912173\") "
Sep 10 13:37:50.002805 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002546    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8612a427-bc5b-4ecd-9ba1-2c9399912173" (UID: "8612a427-bc5b-4ecd-9ba1-2c9399912173"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:37:50.002805 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002660    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-data" (OuterVolumeSpecName: "istio-data") pod "8612a427-bc5b-4ecd-9ba1-2c9399912173" (UID: "8612a427-bc5b-4ecd-9ba1-2c9399912173"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:37:50.002805 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002686    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8612a427-bc5b-4ecd-9ba1-2c9399912173-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8612a427-bc5b-4ecd-9ba1-2c9399912173" (UID: "8612a427-bc5b-4ecd-9ba1-2c9399912173"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:37:50.002996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002801    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8612a427-bc5b-4ecd-9ba1-2c9399912173" (UID: "8612a427-bc5b-4ecd-9ba1-2c9399912173"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:37:50.002996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002920    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8612a427-bc5b-4ecd-9ba1-2c9399912173" (UID: "8612a427-bc5b-4ecd-9ba1-2c9399912173"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:37:50.002996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.002981    2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-data\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:50.003143 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.003000    2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-workload-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:50.003143 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.003014    2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8612a427-bc5b-4ecd-9ba1-2c9399912173-istiod-ca-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:50.003143 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.003026    2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-workload-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:50.004621 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.004551    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8612a427-bc5b-4ecd-9ba1-2c9399912173-kube-api-access-h75wp" (OuterVolumeSpecName: "kube-api-access-h75wp") pod "8612a427-bc5b-4ecd-9ba1-2c9399912173" (UID: "8612a427-bc5b-4ecd-9ba1-2c9399912173"). InnerVolumeSpecName "kube-api-access-h75wp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:37:50.004710 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.004615    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8612a427-bc5b-4ecd-9ba1-2c9399912173" (UID: "8612a427-bc5b-4ecd-9ba1-2c9399912173"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:37:50.004958 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.004934    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8612a427-bc5b-4ecd-9ba1-2c9399912173" (UID: "8612a427-bc5b-4ecd-9ba1-2c9399912173"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 13:37:50.005050 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.005005    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-token" (OuterVolumeSpecName: "istio-token") pod "8612a427-bc5b-4ecd-9ba1-2c9399912173" (UID: "8612a427-bc5b-4ecd-9ba1-2c9399912173"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:37:50.103556 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.103521    2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-credential-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:50.103556 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.103552    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-token\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:50.103556 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.103561    2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-podinfo\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:50.103753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.103569    2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8612a427-bc5b-4ecd-9ba1-2c9399912173-istio-envoy\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:50.103753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.103593    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h75wp\" (UniqueName: \"kubernetes.io/projected/8612a427-bc5b-4ecd-9ba1-2c9399912173-kube-api-access-h75wp\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:37:50.598415 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.598381    2551 generic.go:356] "Generic (PLEG): container finished" podID="8612a427-bc5b-4ecd-9ba1-2c9399912173" containerID="7fa19316e7a2ee7598a5341a94b222c1b74d206e86b6f2d378560e27f4eb436e" exitCode=0
Sep 10 13:37:50.598554 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.598426    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"
Sep 10 13:37:50.598554 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.598432    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw" event={"ID":"8612a427-bc5b-4ecd-9ba1-2c9399912173","Type":"ContainerDied","Data":"7fa19316e7a2ee7598a5341a94b222c1b74d206e86b6f2d378560e27f4eb436e"}
Sep 10 13:37:50.598554 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.598496    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw" event={"ID":"8612a427-bc5b-4ecd-9ba1-2c9399912173","Type":"ContainerDied","Data":"ca97d5203c2a355e961e5d7d953d22aa4295086ba9cc813477525b0a6d4eb10d"}
Sep 10 13:37:50.598554 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.598517    2551 scope.go:117] "RemoveContainer" containerID="7fa19316e7a2ee7598a5341a94b222c1b74d206e86b6f2d378560e27f4eb436e"
Sep 10 13:37:50.607459 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.607441    2551 scope.go:117] "RemoveContainer" containerID="e90b849bdb8642547685321ca1798d01d02f07570802ca55136fc92cb14b51b1"
Sep 10 13:37:50.614824 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.614805    2551 scope.go:117] "RemoveContainer" containerID="7fa19316e7a2ee7598a5341a94b222c1b74d206e86b6f2d378560e27f4eb436e"
Sep 10 13:37:50.615092 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:37:50.615074    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fa19316e7a2ee7598a5341a94b222c1b74d206e86b6f2d378560e27f4eb436e\": container with ID starting with 7fa19316e7a2ee7598a5341a94b222c1b74d206e86b6f2d378560e27f4eb436e not found: ID does not exist" containerID="7fa19316e7a2ee7598a5341a94b222c1b74d206e86b6f2d378560e27f4eb436e"
Sep 10 13:37:50.615133 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.615100    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fa19316e7a2ee7598a5341a94b222c1b74d206e86b6f2d378560e27f4eb436e"} err="failed to get container status \"7fa19316e7a2ee7598a5341a94b222c1b74d206e86b6f2d378560e27f4eb436e\": rpc error: code = NotFound desc = could not find container \"7fa19316e7a2ee7598a5341a94b222c1b74d206e86b6f2d378560e27f4eb436e\": container with ID starting with 7fa19316e7a2ee7598a5341a94b222c1b74d206e86b6f2d378560e27f4eb436e not found: ID does not exist"
Sep 10 13:37:50.615133 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.615116    2551 scope.go:117] "RemoveContainer" containerID="e90b849bdb8642547685321ca1798d01d02f07570802ca55136fc92cb14b51b1"
Sep 10 13:37:50.615352 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:37:50.615339    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e90b849bdb8642547685321ca1798d01d02f07570802ca55136fc92cb14b51b1\": container with ID starting with e90b849bdb8642547685321ca1798d01d02f07570802ca55136fc92cb14b51b1 not found: ID does not exist" containerID="e90b849bdb8642547685321ca1798d01d02f07570802ca55136fc92cb14b51b1"
Sep 10 13:37:50.615388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.615355    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e90b849bdb8642547685321ca1798d01d02f07570802ca55136fc92cb14b51b1"} err="failed to get container status \"e90b849bdb8642547685321ca1798d01d02f07570802ca55136fc92cb14b51b1\": rpc error: code = NotFound desc = could not find container \"e90b849bdb8642547685321ca1798d01d02f07570802ca55136fc92cb14b51b1\": container with ID starting with e90b849bdb8642547685321ca1798d01d02f07570802ca55136fc92cb14b51b1 not found: ID does not exist"
Sep 10 13:37:50.624805 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.624780    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"]
Sep 10 13:37:50.630904 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:50.630884    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-3c32b1-openshift-default-74946dd4f6-tjqxw"]
Sep 10 13:37:51.709539 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:51.709503    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8612a427-bc5b-4ecd-9ba1-2c9399912173" path="/var/lib/kubelet/pods/8612a427-bc5b-4ecd-9ba1-2c9399912173/volumes"
Sep 10 13:37:55.070805 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:37:55.070765    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:38:02.781224 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.781192    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"]
Sep 10 13:38:02.781562 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.781551    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8612a427-bc5b-4ecd-9ba1-2c9399912173" containerName="istio-proxy"
Sep 10 13:38:02.781562 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.781563    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8612a427-bc5b-4ecd-9ba1-2c9399912173" containerName="istio-proxy"
Sep 10 13:38:02.781660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.781586    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89b0fe9c-7c07-4be0-a391-6bb97b49d0c9" containerName="revoke-keys"
Sep 10 13:38:02.781660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.781593    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="89b0fe9c-7c07-4be0-a391-6bb97b49d0c9" containerName="revoke-keys"
Sep 10 13:38:02.781660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.781602    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8612a427-bc5b-4ecd-9ba1-2c9399912173" containerName="istio-proxy"
Sep 10 13:38:02.781660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.781608    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8612a427-bc5b-4ecd-9ba1-2c9399912173" containerName="istio-proxy"
Sep 10 13:38:02.781660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.781622    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1ede543-04c8-4c3d-a276-92c260c2b430" containerName="main"
Sep 10 13:38:02.781660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.781627    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1ede543-04c8-4c3d-a276-92c260c2b430" containerName="main"
Sep 10 13:38:02.781837 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.781703    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8612a427-bc5b-4ecd-9ba1-2c9399912173" containerName="istio-proxy"
Sep 10 13:38:02.781837 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.781712    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="89b0fe9c-7c07-4be0-a391-6bb97b49d0c9" containerName="revoke-keys"
Sep 10 13:38:02.781837 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.781720    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1ede543-04c8-4c3d-a276-92c260c2b430" containerName="main"
Sep 10 13:38:02.786172 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.786149    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:02.789892 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.789868    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-198b30\"/\"noten-model-198b30-kserve-self-signed-certs\""
Sep 10 13:38:02.790001 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.789871    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-198b30\"/\"openshift-service-ca.crt\""
Sep 10 13:38:02.790001 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.789869    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-198b30\"/\"default-dockercfg-f4x6v\""
Sep 10 13:38:02.790110 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.790027    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-198b30\"/\"kube-root-ca.crt\""
Sep 10 13:38:02.797643 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.797530    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"]
Sep 10 13:38:02.914316 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.914286    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96r9b\" (UniqueName: \"kubernetes.io/projected/86b1820b-1640-444d-b2fd-8c9ae65bf5be-kube-api-access-96r9b\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:02.914475 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.914337    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-home\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:02.914475 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.914434    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-model-cache\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:02.914475 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.914463    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-tmp-dir\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:02.914676 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.914494    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-dshm\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:02.914676 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:02.914547    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b1820b-1640-444d-b2fd-8c9ae65bf5be-tls-certs\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:03.015775 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.015743    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96r9b\" (UniqueName: \"kubernetes.io/projected/86b1820b-1640-444d-b2fd-8c9ae65bf5be-kube-api-access-96r9b\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:03.015955 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.015800    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-home\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:03.015955 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.015849    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-model-cache\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:03.015955 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.015864    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-tmp-dir\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:03.015955 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.015890    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-dshm\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:03.015955 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.015916    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b1820b-1640-444d-b2fd-8c9ae65bf5be-tls-certs\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:03.016200 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.016166    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-home\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:03.016317 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.016286    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-tmp-dir\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:03.016376 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.016320    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-model-cache\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:03.018042 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.018024    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-dshm\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:03.018286 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.018270    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b1820b-1640-444d-b2fd-8c9ae65bf5be-tls-certs\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:03.026484 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.026460    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96r9b\" (UniqueName: \"kubernetes.io/projected/86b1820b-1640-444d-b2fd-8c9ae65bf5be-kube-api-access-96r9b\") pod \"noten-model-198b30-kserve-5995b66c5-gn4wz\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") " pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:03.095549 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.095460    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:03.368281 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.368257    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"]
Sep 10 13:38:03.369815 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:38:03.369788    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86b1820b_1640_444d_b2fd_8c9ae65bf5be.slice/crio-26a4f34ce065255ebb8cc2865dba5e4d006ed93b9102cb3a29869c6ba4807408 WatchSource:0}: Error finding container 26a4f34ce065255ebb8cc2865dba5e4d006ed93b9102cb3a29869c6ba4807408: Status 404 returned error can't find the container with id 26a4f34ce065255ebb8cc2865dba5e4d006ed93b9102cb3a29869c6ba4807408
Sep 10 13:38:03.662991 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.662904    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz" event={"ID":"86b1820b-1640-444d-b2fd-8c9ae65bf5be","Type":"ContainerStarted","Data":"57ead89f96dab026ad6e0bf844ccd904800bb210750a1291f3d767b9b96ab92d"}
Sep 10 13:38:03.662991 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.662951    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz" event={"ID":"86b1820b-1640-444d-b2fd-8c9ae65bf5be","Type":"ContainerStarted","Data":"26a4f34ce065255ebb8cc2865dba5e4d006ed93b9102cb3a29869c6ba4807408"}
Sep 10 13:38:03.685331 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:03.685286    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz" podStartSLOduration=1.6852723059999999 podStartE2EDuration="1.685272306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:38:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:38:03.684480658 +0000 UTC m=+1194.503519059" watchObservedRunningTime="2026-09-10 13:38:03.685272306 +0000 UTC m=+1194.504310705"
Sep 10 13:38:09.680557 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:09.680529    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:38:09.681917 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:09.681894    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:38:11.282545 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:11.282508    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:38:13.095950 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:13.095916    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:13.096335 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:13.095967    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:13.108464 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:13.108436    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:13.723569 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:13.723538    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:18.931008 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:18.930978    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"]
Sep 10 13:38:22.775621 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:22.775586    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:38:33.950162 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:33.950121    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz" podUID="86b1820b-1640-444d-b2fd-8c9ae65bf5be" containerName="main" containerID="cri-o://57ead89f96dab026ad6e0bf844ccd904800bb210750a1291f3d767b9b96ab92d" gracePeriod=45
Sep 10 13:38:34.195972 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.195945    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:34.291051 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.291020    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-dshm\") pod \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") "
Sep 10 13:38:34.291216 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.291064    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b1820b-1640-444d-b2fd-8c9ae65bf5be-tls-certs\") pod \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") "
Sep 10 13:38:34.291216 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.291097    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-model-cache\") pod \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") "
Sep 10 13:38:34.291216 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.291154    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96r9b\" (UniqueName: \"kubernetes.io/projected/86b1820b-1640-444d-b2fd-8c9ae65bf5be-kube-api-access-96r9b\") pod \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") "
Sep 10 13:38:34.291369 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.291345    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-tmp-dir\") pod \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") "
Sep 10 13:38:34.291436 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.291414    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-model-cache" (OuterVolumeSpecName: "model-cache") pod "86b1820b-1640-444d-b2fd-8c9ae65bf5be" (UID: "86b1820b-1640-444d-b2fd-8c9ae65bf5be"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:34.291502 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.291435    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-home\") pod \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\" (UID: \"86b1820b-1640-444d-b2fd-8c9ae65bf5be\") "
Sep 10 13:38:34.291679 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.291626    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "86b1820b-1640-444d-b2fd-8c9ae65bf5be" (UID: "86b1820b-1640-444d-b2fd-8c9ae65bf5be"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:34.291784 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.291705    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-home" (OuterVolumeSpecName: "home") pod "86b1820b-1640-444d-b2fd-8c9ae65bf5be" (UID: "86b1820b-1640-444d-b2fd-8c9ae65bf5be"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:34.291906 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.291844    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-tmp-dir\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:38:34.291906 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.291870    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-home\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:38:34.291906 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.291884    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-model-cache\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:38:34.293336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.293314    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-dshm" (OuterVolumeSpecName: "dshm") pod "86b1820b-1640-444d-b2fd-8c9ae65bf5be" (UID: "86b1820b-1640-444d-b2fd-8c9ae65bf5be"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:34.293666 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.293647    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86b1820b-1640-444d-b2fd-8c9ae65bf5be-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "86b1820b-1640-444d-b2fd-8c9ae65bf5be" (UID: "86b1820b-1640-444d-b2fd-8c9ae65bf5be"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:38:34.293728 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.293671    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86b1820b-1640-444d-b2fd-8c9ae65bf5be-kube-api-access-96r9b" (OuterVolumeSpecName: "kube-api-access-96r9b") pod "86b1820b-1640-444d-b2fd-8c9ae65bf5be" (UID: "86b1820b-1640-444d-b2fd-8c9ae65bf5be"). InnerVolumeSpecName "kube-api-access-96r9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:38:34.393159 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.393122    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86b1820b-1640-444d-b2fd-8c9ae65bf5be-dshm\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:38:34.393159 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.393157    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b1820b-1640-444d-b2fd-8c9ae65bf5be-tls-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:38:34.393159 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.393167    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96r9b\" (UniqueName: \"kubernetes.io/projected/86b1820b-1640-444d-b2fd-8c9ae65bf5be-kube-api-access-96r9b\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:38:34.823164 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.823129    2551 generic.go:356] "Generic (PLEG): container finished" podID="86b1820b-1640-444d-b2fd-8c9ae65bf5be" containerID="57ead89f96dab026ad6e0bf844ccd904800bb210750a1291f3d767b9b96ab92d" exitCode=0
Sep 10 13:38:34.823320 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.823167    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"
Sep 10 13:38:34.823320 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.823213    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz" event={"ID":"86b1820b-1640-444d-b2fd-8c9ae65bf5be","Type":"ContainerDied","Data":"57ead89f96dab026ad6e0bf844ccd904800bb210750a1291f3d767b9b96ab92d"}
Sep 10 13:38:34.823320 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.823255    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz" event={"ID":"86b1820b-1640-444d-b2fd-8c9ae65bf5be","Type":"ContainerDied","Data":"26a4f34ce065255ebb8cc2865dba5e4d006ed93b9102cb3a29869c6ba4807408"}
Sep 10 13:38:34.823320 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.823272    2551 scope.go:117] "RemoveContainer" containerID="57ead89f96dab026ad6e0bf844ccd904800bb210750a1291f3d767b9b96ab92d"
Sep 10 13:38:34.832102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.832086    2551 scope.go:117] "RemoveContainer" containerID="57ead89f96dab026ad6e0bf844ccd904800bb210750a1291f3d767b9b96ab92d"
Sep 10 13:38:34.832373 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:38:34.832355    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57ead89f96dab026ad6e0bf844ccd904800bb210750a1291f3d767b9b96ab92d\": container with ID starting with 57ead89f96dab026ad6e0bf844ccd904800bb210750a1291f3d767b9b96ab92d not found: ID does not exist" containerID="57ead89f96dab026ad6e0bf844ccd904800bb210750a1291f3d767b9b96ab92d"
Sep 10 13:38:34.832418 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.832380    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57ead89f96dab026ad6e0bf844ccd904800bb210750a1291f3d767b9b96ab92d"} err="failed to get container status \"57ead89f96dab026ad6e0bf844ccd904800bb210750a1291f3d767b9b96ab92d\": rpc error: code = NotFound desc = could not find container \"57ead89f96dab026ad6e0bf844ccd904800bb210750a1291f3d767b9b96ab92d\": container with ID starting with 57ead89f96dab026ad6e0bf844ccd904800bb210750a1291f3d767b9b96ab92d not found: ID does not exist"
Sep 10 13:38:34.853471 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.853447    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"]
Sep 10 13:38:34.863183 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:34.863157    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-198b30/noten-model-198b30-kserve-5995b66c5-gn4wz"]
Sep 10 13:38:35.710484 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:35.710438    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86b1820b-1640-444d-b2fd-8c9ae65bf5be" path="/var/lib/kubelet/pods/86b1820b-1640-444d-b2fd-8c9ae65bf5be/volumes"
Sep 10 13:38:36.760563 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:36.760535    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"]
Sep 10 13:38:40.075372 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:40.075342    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:38:51.779553 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:51.779491    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4" podUID="39e46455-71e4-4d3f-9d9b-04b2b6006c9e" containerName="main" containerID="cri-o://84588c5fa58f77d85af4d2db6aa136ad8607a805b946f9efc97f4ddc902b9a1e" gracePeriod=45
Sep 10 13:38:51.911154 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:51.911123    2551 generic.go:356] "Generic (PLEG): container finished" podID="39e46455-71e4-4d3f-9d9b-04b2b6006c9e" containerID="84588c5fa58f77d85af4d2db6aa136ad8607a805b946f9efc97f4ddc902b9a1e" exitCode=0
Sep 10 13:38:51.911289 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:51.911179    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4" event={"ID":"39e46455-71e4-4d3f-9d9b-04b2b6006c9e","Type":"ContainerDied","Data":"84588c5fa58f77d85af4d2db6aa136ad8607a805b946f9efc97f4ddc902b9a1e"}
Sep 10 13:38:52.019270 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.019247    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:38:52.150042 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.149961    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-dshm\") pod \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") "
Sep 10 13:38:52.150042 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.149998    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-tls-certs\") pod \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") "
Sep 10 13:38:52.150042 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.150039    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-home\") pod \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") "
Sep 10 13:38:52.150287 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.150069    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-model-cache\") pod \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") "
Sep 10 13:38:52.150287 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.150100    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcdb5\" (UniqueName: \"kubernetes.io/projected/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-kube-api-access-fcdb5\") pod \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") "
Sep 10 13:38:52.150287 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.150217    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-tmp-dir\") pod \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\" (UID: \"39e46455-71e4-4d3f-9d9b-04b2b6006c9e\") "
Sep 10 13:38:52.150452 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.150332    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-model-cache" (OuterVolumeSpecName: "model-cache") pod "39e46455-71e4-4d3f-9d9b-04b2b6006c9e" (UID: "39e46455-71e4-4d3f-9d9b-04b2b6006c9e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:52.150452 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.150347    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-home" (OuterVolumeSpecName: "home") pod "39e46455-71e4-4d3f-9d9b-04b2b6006c9e" (UID: "39e46455-71e4-4d3f-9d9b-04b2b6006c9e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:52.150563 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.150504    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "39e46455-71e4-4d3f-9d9b-04b2b6006c9e" (UID: "39e46455-71e4-4d3f-9d9b-04b2b6006c9e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:52.150563 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.150538    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-home\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:38:52.150563 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.150550    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-model-cache\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:38:52.152199 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.152181    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-dshm" (OuterVolumeSpecName: "dshm") pod "39e46455-71e4-4d3f-9d9b-04b2b6006c9e" (UID: "39e46455-71e4-4d3f-9d9b-04b2b6006c9e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:52.152480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.152465    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "39e46455-71e4-4d3f-9d9b-04b2b6006c9e" (UID: "39e46455-71e4-4d3f-9d9b-04b2b6006c9e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:38:52.152532 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.152508    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-kube-api-access-fcdb5" (OuterVolumeSpecName: "kube-api-access-fcdb5") pod "39e46455-71e4-4d3f-9d9b-04b2b6006c9e" (UID: "39e46455-71e4-4d3f-9d9b-04b2b6006c9e"). InnerVolumeSpecName "kube-api-access-fcdb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:38:52.251060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.251032    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-dshm\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:38:52.251060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.251057    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-tls-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:38:52.251218 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.251069    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcdb5\" (UniqueName: \"kubernetes.io/projected/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-kube-api-access-fcdb5\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:38:52.251218 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.251078    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39e46455-71e4-4d3f-9d9b-04b2b6006c9e-tmp-dir\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:38:52.282359 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.282334    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:38:52.917168 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.917137    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"
Sep 10 13:38:52.917622 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.917174    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4" event={"ID":"39e46455-71e4-4d3f-9d9b-04b2b6006c9e","Type":"ContainerDied","Data":"a7972828faffbdbbfe26526694d5be99f3375944bae037fe917ed21d405f2e2b"}
Sep 10 13:38:52.917622 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.917227    2551 scope.go:117] "RemoveContainer" containerID="84588c5fa58f77d85af4d2db6aa136ad8607a805b946f9efc97f4ddc902b9a1e"
Sep 10 13:38:52.940073 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.940047    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"]
Sep 10 13:38:52.946086 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:52.946063    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-d6159c/test-model-d6159c-kserve-7c454885b5-v4nx4"]
Sep 10 13:38:53.709818 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:38:53.709784    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39e46455-71e4-4d3f-9d9b-04b2b6006c9e" path="/var/lib/kubelet/pods/39e46455-71e4-4d3f-9d9b-04b2b6006c9e/volumes"
Sep 10 13:39:02.119177 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:02.119083    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:39:09.299457 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:09.299424    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:39:17.317437 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:17.317403    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:39:18.486421 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:18.486391    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"]
Sep 10 13:39:18.486865 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:18.486606    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l" podUID="1a9d58aa-32da-46cb-9b71-deb195f77b43" containerName="istio-proxy" containerID="cri-o://1e7a46e1b84866cae0537715acc1801ce8a1d8b4c788188e2f1ad285f78d93bf" gracePeriod=30
Sep 10 13:39:23.733524 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.733500    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:39:23.829033 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.828958    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-workload-certs\") pod \"1a9d58aa-32da-46cb-9b71-deb195f77b43\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") "
Sep 10 13:39:23.829033 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829006    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-podinfo\") pod \"1a9d58aa-32da-46cb-9b71-deb195f77b43\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") "
Sep 10 13:39:23.829225 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829038    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a9d58aa-32da-46cb-9b71-deb195f77b43-istiod-ca-cert\") pod \"1a9d58aa-32da-46cb-9b71-deb195f77b43\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") "
Sep 10 13:39:23.829225 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829064    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-token\") pod \"1a9d58aa-32da-46cb-9b71-deb195f77b43\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") "
Sep 10 13:39:23.829225 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829107    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-workload-socket\") pod \"1a9d58aa-32da-46cb-9b71-deb195f77b43\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") "
Sep 10 13:39:23.829225 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829140    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sb5s7\" (UniqueName: \"kubernetes.io/projected/1a9d58aa-32da-46cb-9b71-deb195f77b43-kube-api-access-sb5s7\") pod \"1a9d58aa-32da-46cb-9b71-deb195f77b43\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") "
Sep 10 13:39:23.829225 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829174    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-envoy\") pod \"1a9d58aa-32da-46cb-9b71-deb195f77b43\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") "
Sep 10 13:39:23.829480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829218    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1a9d58aa-32da-46cb-9b71-deb195f77b43" (UID: "1a9d58aa-32da-46cb-9b71-deb195f77b43"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:39:23.829480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829261    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-credential-socket\") pod \"1a9d58aa-32da-46cb-9b71-deb195f77b43\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") "
Sep 10 13:39:23.829480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829298    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-data\") pod \"1a9d58aa-32da-46cb-9b71-deb195f77b43\" (UID: \"1a9d58aa-32da-46cb-9b71-deb195f77b43\") "
Sep 10 13:39:23.829480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829419    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a9d58aa-32da-46cb-9b71-deb195f77b43-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1a9d58aa-32da-46cb-9b71-deb195f77b43" (UID: "1a9d58aa-32da-46cb-9b71-deb195f77b43"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:39:23.829720 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829592    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1a9d58aa-32da-46cb-9b71-deb195f77b43" (UID: "1a9d58aa-32da-46cb-9b71-deb195f77b43"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:39:23.829720 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829666    2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a9d58aa-32da-46cb-9b71-deb195f77b43-istiod-ca-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:39:23.829720 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829686    2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-workload-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:39:23.829720 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829687    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1a9d58aa-32da-46cb-9b71-deb195f77b43" (UID: "1a9d58aa-32da-46cb-9b71-deb195f77b43"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:39:23.829720 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.829691    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-data" (OuterVolumeSpecName: "istio-data") pod "1a9d58aa-32da-46cb-9b71-deb195f77b43" (UID: "1a9d58aa-32da-46cb-9b71-deb195f77b43"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:39:23.831269 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.831244    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-token" (OuterVolumeSpecName: "istio-token") pod "1a9d58aa-32da-46cb-9b71-deb195f77b43" (UID: "1a9d58aa-32da-46cb-9b71-deb195f77b43"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:39:23.831384 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.831266    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1a9d58aa-32da-46cb-9b71-deb195f77b43" (UID: "1a9d58aa-32da-46cb-9b71-deb195f77b43"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 13:39:23.831384 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.831345    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1a9d58aa-32da-46cb-9b71-deb195f77b43" (UID: "1a9d58aa-32da-46cb-9b71-deb195f77b43"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:39:23.831384 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.831352    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a9d58aa-32da-46cb-9b71-deb195f77b43-kube-api-access-sb5s7" (OuterVolumeSpecName: "kube-api-access-sb5s7") pod "1a9d58aa-32da-46cb-9b71-deb195f77b43" (UID: "1a9d58aa-32da-46cb-9b71-deb195f77b43"). InnerVolumeSpecName "kube-api-access-sb5s7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:39:23.930581 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.930557    2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-credential-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:39:23.930709 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.930602    2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-data\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:39:23.930709 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.930617    2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-podinfo\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:39:23.930709 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.930629    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-token\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:39:23.930709 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.930642    2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-workload-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:39:23.930709 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.930656    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sb5s7\" (UniqueName: \"kubernetes.io/projected/1a9d58aa-32da-46cb-9b71-deb195f77b43-kube-api-access-sb5s7\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:39:23.930709 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:23.930669    2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a9d58aa-32da-46cb-9b71-deb195f77b43-istio-envoy\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:39:24.076942 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:24.076904    2551 generic.go:356] "Generic (PLEG): container finished" podID="1a9d58aa-32da-46cb-9b71-deb195f77b43" containerID="1e7a46e1b84866cae0537715acc1801ce8a1d8b4c788188e2f1ad285f78d93bf" exitCode=0
Sep 10 13:39:24.077081 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:24.076961    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l" event={"ID":"1a9d58aa-32da-46cb-9b71-deb195f77b43","Type":"ContainerDied","Data":"1e7a46e1b84866cae0537715acc1801ce8a1d8b4c788188e2f1ad285f78d93bf"}
Sep 10 13:39:24.077081 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:24.076964    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"
Sep 10 13:39:24.077081 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:24.077029    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l" event={"ID":"1a9d58aa-32da-46cb-9b71-deb195f77b43","Type":"ContainerDied","Data":"60cb3e32010742eb383122c5c877f97b325c0c25e58483180756378371f81bbe"}
Sep 10 13:39:24.077081 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:24.077052    2551 scope.go:117] "RemoveContainer" containerID="1e7a46e1b84866cae0537715acc1801ce8a1d8b4c788188e2f1ad285f78d93bf"
Sep 10 13:39:24.086077 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:24.086057    2551 scope.go:117] "RemoveContainer" containerID="10f20b456f5fb6f71e64518b616528b39ef13d7ef4b58abcb929504dee0414c3"
Sep 10 13:39:24.093439 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:24.093420    2551 scope.go:117] "RemoveContainer" containerID="1e7a46e1b84866cae0537715acc1801ce8a1d8b4c788188e2f1ad285f78d93bf"
Sep 10 13:39:24.093732 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:39:24.093713    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e7a46e1b84866cae0537715acc1801ce8a1d8b4c788188e2f1ad285f78d93bf\": container with ID starting with 1e7a46e1b84866cae0537715acc1801ce8a1d8b4c788188e2f1ad285f78d93bf not found: ID does not exist" containerID="1e7a46e1b84866cae0537715acc1801ce8a1d8b4c788188e2f1ad285f78d93bf"
Sep 10 13:39:24.093793 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:24.093738    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e7a46e1b84866cae0537715acc1801ce8a1d8b4c788188e2f1ad285f78d93bf"} err="failed to get container status \"1e7a46e1b84866cae0537715acc1801ce8a1d8b4c788188e2f1ad285f78d93bf\": rpc error: code = NotFound desc = could not find container \"1e7a46e1b84866cae0537715acc1801ce8a1d8b4c788188e2f1ad285f78d93bf\": container with ID starting with 1e7a46e1b84866cae0537715acc1801ce8a1d8b4c788188e2f1ad285f78d93bf not found: ID does not exist"
Sep 10 13:39:24.093793 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:24.093754    2551 scope.go:117] "RemoveContainer" containerID="10f20b456f5fb6f71e64518b616528b39ef13d7ef4b58abcb929504dee0414c3"
Sep 10 13:39:24.094022 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:39:24.094004    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10f20b456f5fb6f71e64518b616528b39ef13d7ef4b58abcb929504dee0414c3\": container with ID starting with 10f20b456f5fb6f71e64518b616528b39ef13d7ef4b58abcb929504dee0414c3 not found: ID does not exist" containerID="10f20b456f5fb6f71e64518b616528b39ef13d7ef4b58abcb929504dee0414c3"
Sep 10 13:39:24.094110 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:24.094027    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10f20b456f5fb6f71e64518b616528b39ef13d7ef4b58abcb929504dee0414c3"} err="failed to get container status \"10f20b456f5fb6f71e64518b616528b39ef13d7ef4b58abcb929504dee0414c3\": rpc error: code = NotFound desc = could not find container \"10f20b456f5fb6f71e64518b616528b39ef13d7ef4b58abcb929504dee0414c3\": container with ID starting with 10f20b456f5fb6f71e64518b616528b39ef13d7ef4b58abcb929504dee0414c3 not found: ID does not exist"
Sep 10 13:39:24.102961 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:24.102941    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"]
Sep 10 13:39:24.109168 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:24.109147    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d6159c-openshift-default-754586568-6b69l"]
Sep 10 13:39:25.710299 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:25.710266    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a9d58aa-32da-46cb-9b71-deb195f77b43" path="/var/lib/kubelet/pods/1a9d58aa-32da-46cb-9b71-deb195f77b43/volumes"
Sep 10 13:39:28.080847 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:28.080817    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:39:48.807961 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.807931    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"]
Sep 10 13:39:48.808388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.808252    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86b1820b-1640-444d-b2fd-8c9ae65bf5be" containerName="main"
Sep 10 13:39:48.808388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.808264    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="86b1820b-1640-444d-b2fd-8c9ae65bf5be" containerName="main"
Sep 10 13:39:48.808388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.808281    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a9d58aa-32da-46cb-9b71-deb195f77b43" containerName="istio-proxy"
Sep 10 13:39:48.808388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.808287    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a9d58aa-32da-46cb-9b71-deb195f77b43" containerName="istio-proxy"
Sep 10 13:39:48.808388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.808296    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a9d58aa-32da-46cb-9b71-deb195f77b43" containerName="istio-proxy"
Sep 10 13:39:48.808388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.808301    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a9d58aa-32da-46cb-9b71-deb195f77b43" containerName="istio-proxy"
Sep 10 13:39:48.808388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.808315    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e46455-71e4-4d3f-9d9b-04b2b6006c9e" containerName="main"
Sep 10 13:39:48.808388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.808321    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e46455-71e4-4d3f-9d9b-04b2b6006c9e" containerName="main"
Sep 10 13:39:48.808388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.808369    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8612a427-bc5b-4ecd-9ba1-2c9399912173" containerName="istio-proxy"
Sep 10 13:39:48.808388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.808377    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a9d58aa-32da-46cb-9b71-deb195f77b43" containerName="istio-proxy"
Sep 10 13:39:48.808388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.808383    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="86b1820b-1640-444d-b2fd-8c9ae65bf5be" containerName="main"
Sep 10 13:39:48.808388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.808389    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e46455-71e4-4d3f-9d9b-04b2b6006c9e" containerName="main"
Sep 10 13:39:48.811367 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.811351    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:48.814804 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.814781    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-89be92-openshift-default-dockercfg-k4429\""
Sep 10 13:39:48.855255 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.855228    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"]
Sep 10 13:39:48.957379 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.957343    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-workload-socket\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:48.957379 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.957383    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-token\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:48.957623 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.957410    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-podinfo\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:48.957623 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.957481    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-envoy\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:48.957623 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.957559    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v4bj\" (UniqueName: \"kubernetes.io/projected/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-kube-api-access-6v4bj\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:48.957736 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.957640    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istiod-ca-cert\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:48.957736 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.957664    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-data\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:48.957736 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.957713    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-workload-certs\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:48.957833 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:48.957754    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-credential-socket\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.058879 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.058773    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-workload-certs\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.058879 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.058831    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-credential-socket\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.059109 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.058950    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-workload-socket\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.059109 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.058992    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-token\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.059109 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.059021    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-podinfo\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.059109 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.059054    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-envoy\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.059308 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.059111    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v4bj\" (UniqueName: \"kubernetes.io/projected/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-kube-api-access-6v4bj\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.059308 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.059152    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-credential-socket\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.059308 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.059203    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istiod-ca-cert\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.059308 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.059208    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-workload-certs\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.059308 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.059252    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-data\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.059308 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.059294    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-workload-socket\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.059609 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.059565    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-data\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.059829 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.059812    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istiod-ca-cert\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.061354 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.061337    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-envoy\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.061433 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.061416    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-podinfo\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.072958 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.072934    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-token\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.073051 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.073005    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v4bj\" (UniqueName: \"kubernetes.io/projected/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-kube-api-access-6v4bj\") pod \"e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") " pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.120845 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.120811    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:49.402304 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.402283    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"]
Sep 10 13:39:49.403778 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:39:49.403752    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3507b40_69d2_4c9a_9bbf_18c572d5e7e1.slice/crio-2446af26344481cc1ed5c91cfec004bdc621cb9f9d34d3355aea60d4659640a9 WatchSource:0}: Error finding container 2446af26344481cc1ed5c91cfec004bdc621cb9f9d34d3355aea60d4659640a9: Status 404 returned error can't find the container with id 2446af26344481cc1ed5c91cfec004bdc621cb9f9d34d3355aea60d4659640a9
Sep 10 13:39:49.406022 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.405991    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:39:49.406088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.406069    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:39:49.406130 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:49.406102    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:39:50.208432 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:50.208399    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" event={"ID":"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1","Type":"ContainerStarted","Data":"4d9d23f72bbb5813b3f079d03c1fd7e7bc449060180a010fc0390f59317686d7"}
Sep 10 13:39:50.208836 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:50.208452    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" event={"ID":"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1","Type":"ContainerStarted","Data":"2446af26344481cc1ed5c91cfec004bdc621cb9f9d34d3355aea60d4659640a9"}
Sep 10 13:39:50.238969 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:50.238919    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" podStartSLOduration=2.238903974 podStartE2EDuration="2.238903974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:39:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:39:50.236600431 +0000 UTC m=+1301.055638825" watchObservedRunningTime="2026-09-10 13:39:50.238903974 +0000 UTC m=+1301.057942375"
Sep 10 13:39:51.121549 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:51.121511    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:51.126325 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:51.126298    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:51.213284 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:51.213260    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:51.214173 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:51.214154    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:39:56.856016 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:56.855986    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg"]
Sep 10 13:39:56.856418 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:56.856404    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a9d58aa-32da-46cb-9b71-deb195f77b43" containerName="istio-proxy"
Sep 10 13:39:56.859544 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:56.859528    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg"
Sep 10 13:39:56.864782 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:56.864766    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:39:56.864894 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:56.864811    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:39:56.873742 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:56.873716    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg"]
Sep 10 13:39:56.936504 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:56.936475    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25f1de68-f558-4e6c-ab1b-e38639bc598f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg\" (UID: \"25f1de68-f558-4e6c-ab1b-e38639bc598f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg"
Sep 10 13:39:57.037644 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:57.037617    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25f1de68-f558-4e6c-ab1b-e38639bc598f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg\" (UID: \"25f1de68-f558-4e6c-ab1b-e38639bc598f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg"
Sep 10 13:39:57.038197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:57.038177    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25f1de68-f558-4e6c-ab1b-e38639bc598f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg\" (UID: \"25f1de68-f558-4e6c-ab1b-e38639bc598f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg"
Sep 10 13:39:57.169398 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:57.169330    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg"
Sep 10 13:39:57.446483 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:57.446458    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg"]
Sep 10 13:39:57.448569 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:39:57.448541    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25f1de68_f558_4e6c_ab1b_e38639bc598f.slice/crio-313946ecccac323f20106d7f83c752a717a553fe1673bedb95c3afa8b3a39a8a WatchSource:0}: Error finding container 313946ecccac323f20106d7f83c752a717a553fe1673bedb95c3afa8b3a39a8a: Status 404 returned error can't find the container with id 313946ecccac323f20106d7f83c752a717a553fe1673bedb95c3afa8b3a39a8a
Sep 10 13:39:58.248076 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:58.248034    2551 generic.go:356] "Generic (PLEG): container finished" podID="25f1de68-f558-4e6c-ab1b-e38639bc598f" containerID="236f91cca050249b266a1bd275b7a8ee1a5f6bc083087b10cac35fe80fd1a43e" exitCode=0
Sep 10 13:39:58.248441 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:58.248119    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg" event={"ID":"25f1de68-f558-4e6c-ab1b-e38639bc598f","Type":"ContainerDied","Data":"236f91cca050249b266a1bd275b7a8ee1a5f6bc083087b10cac35fe80fd1a43e"}
Sep 10 13:39:58.248441 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:58.248176    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg" event={"ID":"25f1de68-f558-4e6c-ab1b-e38639bc598f","Type":"ContainerStarted","Data":"313946ecccac323f20106d7f83c752a717a553fe1673bedb95c3afa8b3a39a8a"}
Sep 10 13:39:58.485222 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:58.485193    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:39:59.379668 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:59.379645    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg"
Sep 10 13:39:59.458228 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:59.458206    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25f1de68-f558-4e6c-ab1b-e38639bc598f-maas-api-service-ca\") pod \"25f1de68-f558-4e6c-ab1b-e38639bc598f\" (UID: \"25f1de68-f558-4e6c-ab1b-e38639bc598f\") "
Sep 10 13:39:59.458639 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:59.458612    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25f1de68-f558-4e6c-ab1b-e38639bc598f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "25f1de68-f558-4e6c-ab1b-e38639bc598f" (UID: "25f1de68-f558-4e6c-ab1b-e38639bc598f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:39:59.559217 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:39:59.559149    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25f1de68-f558-4e6c-ab1b-e38639bc598f-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:40:00.265187 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:00.265151    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg" event={"ID":"25f1de68-f558-4e6c-ab1b-e38639bc598f","Type":"ContainerDied","Data":"313946ecccac323f20106d7f83c752a717a553fe1673bedb95c3afa8b3a39a8a"}
Sep 10 13:40:00.265456 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:00.265439    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="313946ecccac323f20106d7f83c752a717a553fe1673bedb95c3afa8b3a39a8a"
Sep 10 13:40:00.265690 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:00.265670    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg"
Sep 10 13:40:07.255758 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:07.255714    2551 patch_prober.go:28] interesting pod/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 13:40:07.256120 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:07.255783    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 13:40:08.100216 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:08.100171    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg"]
Sep 10 13:40:08.106422 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:08.106392    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-839f01ff-dl9vg"]
Sep 10 13:40:09.320941 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:09.320906    2551 generic.go:356] "Generic (PLEG): container finished" podID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerID="4d9d23f72bbb5813b3f079d03c1fd7e7bc449060180a010fc0390f59317686d7" exitCode=0
Sep 10 13:40:09.321339 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:09.320981    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" event={"ID":"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1","Type":"ContainerDied","Data":"4d9d23f72bbb5813b3f079d03c1fd7e7bc449060180a010fc0390f59317686d7"}
Sep 10 13:40:09.321399 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:09.321346    2551 scope.go:117] "RemoveContainer" containerID="4d9d23f72bbb5813b3f079d03c1fd7e7bc449060180a010fc0390f59317686d7"
Sep 10 13:40:09.322373 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:09.322332    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:40:09.322471 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:09.322417    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:40:09.322471 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:09.322460    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:40:09.711193 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:09.711104    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25f1de68-f558-4e6c-ab1b-e38639bc598f" path="/var/lib/kubelet/pods/25f1de68-f558-4e6c-ab1b-e38639bc598f/volumes"
Sep 10 13:40:10.327518 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:10.327480    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" event={"ID":"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1","Type":"ContainerStarted","Data":"a9c53306c0d2f8269251aad70d3cb71bb6287adb09da79f5824f6cde3801c619"}
Sep 10 13:40:10.999206 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:10.999170    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:40:11.121722 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:11.121687    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:40:11.122008 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:11.121983    2551 patch_prober.go:28] interesting pod/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 10 13:40:11.122065 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:11.122044    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 10 13:40:12.121512 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:12.121480    2551 patch_prober.go:28] interesting pod/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 10 13:40:12.121935 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:12.121534    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 10 13:40:13.121392 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:13.121354    2551 patch_prober.go:28] interesting pod/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 10 13:40:13.121594 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:13.121417    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 10 13:40:14.121965 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:14.121926    2551 patch_prober.go:28] interesting pod/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 10 13:40:14.122354 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:14.121982    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 10 13:40:15.121607 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:15.121551    2551 patch_prober.go:28] interesting pod/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 10 13:40:15.121789 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:15.121627    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 10 13:40:16.168790 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:16.168756    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:40:16.169261 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:16.169062    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:40:16.169832 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:16.169814    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:40:22.779400 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.779370    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"]
Sep 10 13:40:22.780002 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.779889    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25f1de68-f558-4e6c-ab1b-e38639bc598f" containerName="revoke-keys"
Sep 10 13:40:22.780002 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.779909    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="25f1de68-f558-4e6c-ab1b-e38639bc598f" containerName="revoke-keys"
Sep 10 13:40:22.780126 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.780043    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="25f1de68-f558-4e6c-ab1b-e38639bc598f" containerName="revoke-keys"
Sep 10 13:40:22.782951 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.782932    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.785398 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.785379    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-767dce\"/\"default-dockercfg-zvw8j\""
Sep 10 13:40:22.785612 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.785594    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-767dce\"/\"openshift-service-ca.crt\""
Sep 10 13:40:22.786621 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.786604    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-767dce\"/\"auth-test-model-767dce-kserve-self-signed-certs\""
Sep 10 13:40:22.786735 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.786605    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-767dce\"/\"kube-root-ca.crt\""
Sep 10 13:40:22.796593 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.796555    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"]
Sep 10 13:40:22.865927 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.865893    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-dshm\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.866085 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.865944    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb50401b-3ee6-4097-a145-187a8cb40858-tls-certs\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.866085 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.865974    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-home\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.866192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.866112    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdwdz\" (UniqueName: \"kubernetes.io/projected/cb50401b-3ee6-4097-a145-187a8cb40858-kube-api-access-tdwdz\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.866192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.866177    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-tmp-dir\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.866279 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.866210    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-model-cache\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.966984 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.966948    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-dshm\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.966984 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.966985    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb50401b-3ee6-4097-a145-187a8cb40858-tls-certs\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.967238 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.967002    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-home\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.967238 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.967038    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdwdz\" (UniqueName: \"kubernetes.io/projected/cb50401b-3ee6-4097-a145-187a8cb40858-kube-api-access-tdwdz\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.967238 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.967081    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-tmp-dir\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.967238 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.967102    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-model-cache\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.967493 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.967471    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-home\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.967562 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.967515    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-model-cache\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.967631 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.967599    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-tmp-dir\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.969728 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.969703    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-dshm\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.970095 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.970072    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb50401b-3ee6-4097-a145-187a8cb40858-tls-certs\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:22.977362 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:22.977340    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdwdz\" (UniqueName: \"kubernetes.io/projected/cb50401b-3ee6-4097-a145-187a8cb40858-kube-api-access-tdwdz\") pod \"auth-test-model-767dce-kserve-84c74f475f-xn8gs\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") " pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:23.091558 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:23.091483    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:23.371150 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:23.371123    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"]
Sep 10 13:40:23.373287 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:40:23.373259    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb50401b_3ee6_4097_a145_187a8cb40858.slice/crio-0ff85f9e803669a38e56a4c7eb442c446d6f8d78a79f66caed95294c4f569abc WatchSource:0}: Error finding container 0ff85f9e803669a38e56a4c7eb442c446d6f8d78a79f66caed95294c4f569abc: Status 404 returned error can't find the container with id 0ff85f9e803669a38e56a4c7eb442c446d6f8d78a79f66caed95294c4f569abc
Sep 10 13:40:23.399161 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:23.399132    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs" event={"ID":"cb50401b-3ee6-4097-a145-187a8cb40858","Type":"ContainerStarted","Data":"0ff85f9e803669a38e56a4c7eb442c446d6f8d78a79f66caed95294c4f569abc"}
Sep 10 13:40:24.410200 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:24.410160    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs" event={"ID":"cb50401b-3ee6-4097-a145-187a8cb40858","Type":"ContainerStarted","Data":"ec610fe541e9ef50b5238f50ff93b55768cb2d8a0d316bafbe5b6d93c8d3838b"}
Sep 10 13:40:24.442910 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:24.442864    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs" podStartSLOduration=2.442850187 podStartE2EDuration="2.442850187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:40:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:40:24.440092804 +0000 UTC m=+1335.259131197" watchObservedRunningTime="2026-09-10 13:40:24.442850187 +0000 UTC m=+1335.261888587"
Sep 10 13:40:25.982471 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:25.982438    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:40:33.092252 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.092162    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:33.092252 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.092216    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:33.104793 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.104769    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:33.372934 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.372856    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"]
Sep 10 13:40:33.376366 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.376349    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.381050 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.381029    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-5d75ac-openshift-default-dockercfg-k5jhg\""
Sep 10 13:40:33.396084 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.396060    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"]
Sep 10 13:40:33.460407 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.460370    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280f3887-9106-434c-8088-5086ee526ed0-istiod-ca-cert\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.460613 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.460422    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280f3887-9106-434c-8088-5086ee526ed0-istio-token\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.460613 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.460484    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-istio-envoy\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.460613 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.460512    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-credential-socket\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.460613 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.460545    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5k87\" (UniqueName: \"kubernetes.io/projected/280f3887-9106-434c-8088-5086ee526ed0-kube-api-access-t5k87\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.460841 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.460637    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-workload-socket\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.460841 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.460667    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-istio-data\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.460841 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.460710    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-workload-certs\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.460841 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.460762    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280f3887-9106-434c-8088-5086ee526ed0-istio-podinfo\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.471236 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.471207    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:40:33.562121 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.562089    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280f3887-9106-434c-8088-5086ee526ed0-istio-podinfo\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.562270 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.562166    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280f3887-9106-434c-8088-5086ee526ed0-istiod-ca-cert\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.562371 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.562349    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280f3887-9106-434c-8088-5086ee526ed0-istio-token\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.562437 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.562420    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-istio-envoy\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.562477 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.562460    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-credential-socket\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.562522 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.562500    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5k87\" (UniqueName: \"kubernetes.io/projected/280f3887-9106-434c-8088-5086ee526ed0-kube-api-access-t5k87\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.562596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.562545    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-workload-socket\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.562596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.562569    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-istio-data\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.562702 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.562622    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-workload-certs\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.562876 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.562848    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-credential-socket\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.562876 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.562866    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280f3887-9106-434c-8088-5086ee526ed0-istiod-ca-cert\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.563010 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.562966    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-istio-data\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.563066 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.563024    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-workload-certs\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.563369 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.563339    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-workload-socket\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.564683 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.564665    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280f3887-9106-434c-8088-5086ee526ed0-istio-podinfo\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.564796 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.564755    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-istio-envoy\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.573679 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.573648    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280f3887-9106-434c-8088-5086ee526ed0-istio-token\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.573947 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.573929    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5k87\" (UniqueName: \"kubernetes.io/projected/280f3887-9106-434c-8088-5086ee526ed0-kube-api-access-t5k87\") pod \"e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") " pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.686747 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.686667    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:33.970492 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.970413    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"]
Sep 10 13:40:33.973545 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:40:33.973507    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod280f3887_9106_434c_8088_5086ee526ed0.slice/crio-46a4556b8c865f48ea35f747d1b2a6665825bc96a96098d029a66c99d7ffda47 WatchSource:0}: Error finding container 46a4556b8c865f48ea35f747d1b2a6665825bc96a96098d029a66c99d7ffda47: Status 404 returned error can't find the container with id 46a4556b8c865f48ea35f747d1b2a6665825bc96a96098d029a66c99d7ffda47
Sep 10 13:40:33.975617 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.975563    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:40:33.975688 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.975649    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:40:33.975688 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:33.975678    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:40:34.465938 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:34.465906    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f" event={"ID":"280f3887-9106-434c-8088-5086ee526ed0","Type":"ContainerStarted","Data":"92d97800d426c43fed1c6b7343f4ed3c1c6fc91e28f4adbee4d748208ec4ea0a"}
Sep 10 13:40:34.466330 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:34.465964    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f" event={"ID":"280f3887-9106-434c-8088-5086ee526ed0","Type":"ContainerStarted","Data":"46a4556b8c865f48ea35f747d1b2a6665825bc96a96098d029a66c99d7ffda47"}
Sep 10 13:40:34.504921 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:34.504869    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f" podStartSLOduration=1.504847409 podStartE2EDuration="1.504847409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:40:34.502927301 +0000 UTC m=+1345.321965701" watchObservedRunningTime="2026-09-10 13:40:34.504847409 +0000 UTC m=+1345.323885812"
Sep 10 13:40:34.686940 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:34.686911    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:35.692588 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:35.692545    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:36.477673 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:36.477644    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:36.478618 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:36.478593    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:43.177634 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:43.177595    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:40:52.519740 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:52.519687    2551 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 13:40:52.520228 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:52.519758    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f" podUID="280f3887-9106-434c-8088-5086ee526ed0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 13:40:53.564952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:53.564919    2551 generic.go:356] "Generic (PLEG): container finished" podID="280f3887-9106-434c-8088-5086ee526ed0" containerID="92d97800d426c43fed1c6b7343f4ed3c1c6fc91e28f4adbee4d748208ec4ea0a" exitCode=0
Sep 10 13:40:53.565356 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:53.564995    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f" event={"ID":"280f3887-9106-434c-8088-5086ee526ed0","Type":"ContainerDied","Data":"92d97800d426c43fed1c6b7343f4ed3c1c6fc91e28f4adbee4d748208ec4ea0a"}
Sep 10 13:40:53.565442 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:53.565427    2551 scope.go:117] "RemoveContainer" containerID="92d97800d426c43fed1c6b7343f4ed3c1c6fc91e28f4adbee4d748208ec4ea0a"
Sep 10 13:40:53.566471 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:53.566442    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:40:53.566530 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:53.566515    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:40:53.566567 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:53.566550    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:40:54.573724 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:54.573691    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f" event={"ID":"280f3887-9106-434c-8088-5086ee526ed0","Type":"ContainerStarted","Data":"5e49fd0973a3574729b761487f066a1fd847b4352d4adf3d46fa7fb6c1ae370b"}
Sep 10 13:40:54.687055 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:54.687012    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:54.687386 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:54.687362    2551 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 10 13:40:54.687437 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:54.687416    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f" podUID="280f3887-9106-434c-8088-5086ee526ed0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 10 13:40:55.687810 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:55.687777    2551 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 10 13:40:55.688279 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:55.687829    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f" podUID="280f3887-9106-434c-8088-5086ee526ed0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 10 13:40:56.687701 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:56.687665    2551 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 10 13:40:56.687880 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:56.687721    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f" podUID="280f3887-9106-434c-8088-5086ee526ed0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 10 13:40:57.687840 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:57.687801    2551 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 10 13:40:57.688024 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:57.687868    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f" podUID="280f3887-9106-434c-8088-5086ee526ed0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 10 13:40:58.687494 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:58.687454    2551 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 10 13:40:58.687692 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:58.687532    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f" podUID="280f3887-9106-434c-8088-5086ee526ed0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 10 13:40:59.690842 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:59.690814    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:59.691210 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:59.691091    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:40:59.691752 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:40:59.691733    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:41:05.995072 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:05.995040    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"]
Sep 10 13:41:05.998435 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:05.998416    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.002462 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.002440    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-89be92\"/\"kube-root-ca.crt\""
Sep 10 13:41:06.002602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.002440    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-89be92\"/\"openshift-service-ca.crt\""
Sep 10 13:41:06.002602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.002441    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-89be92\"/\"auth-test-model-89be92-kserve-self-signed-certs\""
Sep 10 13:41:06.002602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.002527    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-89be92\"/\"default-dockercfg-2ldvq\""
Sep 10 13:41:06.012100 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.012076    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"]
Sep 10 13:41:06.058983 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.058960    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c216272e-cb6f-481d-ab3f-7983dfc1939b-tls-certs\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.059089 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.059018    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-home\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.059134 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.059090    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsmcs\" (UniqueName: \"kubernetes.io/projected/c216272e-cb6f-481d-ab3f-7983dfc1939b-kube-api-access-vsmcs\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.059174 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.059143    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-model-cache\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.059212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.059190    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-tmp-dir\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.059257 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.059237    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-dshm\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.160239 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.160212    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-home\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.160384 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.160248    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsmcs\" (UniqueName: \"kubernetes.io/projected/c216272e-cb6f-481d-ab3f-7983dfc1939b-kube-api-access-vsmcs\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.160384 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.160275    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-model-cache\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.160384 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.160311    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-tmp-dir\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.160384 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.160338    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-dshm\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.160384 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.160362    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c216272e-cb6f-481d-ab3f-7983dfc1939b-tls-certs\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.160704 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.160644    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-home\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.160704 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.160650    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-tmp-dir\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.160830 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.160808    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-model-cache\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.162676 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.162646    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c216272e-cb6f-481d-ab3f-7983dfc1939b-tls-certs\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.162764 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.162677    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-dshm\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.170618 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.170599    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsmcs\" (UniqueName: \"kubernetes.io/projected/c216272e-cb6f-481d-ab3f-7983dfc1939b-kube-api-access-vsmcs\") pod \"auth-test-model-89be92-kserve-56746cdff5-gcv7k\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") " pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.307819 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.307739    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:06.619905 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.619880    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"]
Sep 10 13:41:06.621993 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:41:06.621965    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc216272e_cb6f_481d_ab3f_7983dfc1939b.slice/crio-26ff1fe903c67ce27e53805e9ef0c8f577bded0c4d9cef1a616a1e7d4c3b8e96 WatchSource:0}: Error finding container 26ff1fe903c67ce27e53805e9ef0c8f577bded0c4d9cef1a616a1e7d4c3b8e96: Status 404 returned error can't find the container with id 26ff1fe903c67ce27e53805e9ef0c8f577bded0c4d9cef1a616a1e7d4c3b8e96
Sep 10 13:41:06.644636 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:06.644612    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k" event={"ID":"c216272e-cb6f-481d-ab3f-7983dfc1939b","Type":"ContainerStarted","Data":"26ff1fe903c67ce27e53805e9ef0c8f577bded0c4d9cef1a616a1e7d4c3b8e96"}
Sep 10 13:41:07.651971 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:07.651934    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k" event={"ID":"c216272e-cb6f-481d-ab3f-7983dfc1939b","Type":"ContainerStarted","Data":"3b6d382b1a644a9ef716f32883d292f10e11d67951cd67a445c818282bfbe25c"}
Sep 10 13:41:07.675995 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:07.675851    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k" podStartSLOduration=2.675834053 podStartE2EDuration="2.675834053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:41:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:41:07.675564469 +0000 UTC m=+1378.494602847" watchObservedRunningTime="2026-09-10 13:41:07.675834053 +0000 UTC m=+1378.494872454"
Sep 10 13:41:08.001656 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:08.001612    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:41:16.308327 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:16.308291    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:16.308747 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:16.308346    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:16.320099 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:16.320078    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:16.707371 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:16.707293    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:41:22.973023 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:22.972988    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:41:31.663374 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:31.663342    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7784-455d3a"]
Sep 10 13:41:31.667036 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:31.667019    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-455d3a"
Sep 10 13:41:31.670568 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:31.670552    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lpqwx\""
Sep 10 13:41:31.675790 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:31.675769    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-455d3a"]
Sep 10 13:41:31.787160 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:31.787131    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx7w2\" (UniqueName: \"kubernetes.io/projected/4e3e9bab-0125-42e8-8c1f-57094cde86e2-kube-api-access-bx7w2\") pod \"test-curl-7784-455d3a\" (UID: \"4e3e9bab-0125-42e8-8c1f-57094cde86e2\") " pod="opendatahub/test-curl-7784-455d3a"
Sep 10 13:41:31.887874 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:31.887847    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx7w2\" (UniqueName: \"kubernetes.io/projected/4e3e9bab-0125-42e8-8c1f-57094cde86e2-kube-api-access-bx7w2\") pod \"test-curl-7784-455d3a\" (UID: \"4e3e9bab-0125-42e8-8c1f-57094cde86e2\") " pod="opendatahub/test-curl-7784-455d3a"
Sep 10 13:41:31.897534 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:31.897509    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx7w2\" (UniqueName: \"kubernetes.io/projected/4e3e9bab-0125-42e8-8c1f-57094cde86e2-kube-api-access-bx7w2\") pod \"test-curl-7784-455d3a\" (UID: \"4e3e9bab-0125-42e8-8c1f-57094cde86e2\") " pod="opendatahub/test-curl-7784-455d3a"
Sep 10 13:41:31.976751 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:31.976722    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-455d3a"
Sep 10 13:41:32.248392 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:32.248368    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-455d3a"]
Sep 10 13:41:32.250012 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:41:32.249983    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e3e9bab_0125_42e8_8c1f_57094cde86e2.slice/crio-4638c69c8d841e4bea68eee3f88174385dcfdeed7f063c11376f76eb990ed946 WatchSource:0}: Error finding container 4638c69c8d841e4bea68eee3f88174385dcfdeed7f063c11376f76eb990ed946: Status 404 returned error can't find the container with id 4638c69c8d841e4bea68eee3f88174385dcfdeed7f063c11376f76eb990ed946
Sep 10 13:41:32.773276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:32.773238    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-455d3a" event={"ID":"4e3e9bab-0125-42e8-8c1f-57094cde86e2","Type":"ContainerStarted","Data":"b3b991f1e3d0d3e4e7508bbe1af90bff434354aec84737ef1f2df74de6fe65b4"}
Sep 10 13:41:32.773276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:32.773284    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-455d3a" event={"ID":"4e3e9bab-0125-42e8-8c1f-57094cde86e2","Type":"ContainerStarted","Data":"4638c69c8d841e4bea68eee3f88174385dcfdeed7f063c11376f76eb990ed946"}
Sep 10 13:41:32.797230 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:32.797012    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7784-455d3a" podStartSLOduration=1.796994639 podStartE2EDuration="1.796994639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:41:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:41:32.794524529 +0000 UTC m=+1403.613562929" watchObservedRunningTime="2026-09-10 13:41:32.796994639 +0000 UTC m=+1403.616033039"
Sep 10 13:41:33.453727 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:33.453697    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7784-455d3a"]
Sep 10 13:41:33.462117 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:33.462037    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7784-455d3a"]
Sep 10 13:41:33.589686 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:33.589652    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7784-398268"]
Sep 10 13:41:33.590099 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:33.590080    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e3e9bab-0125-42e8-8c1f-57094cde86e2" containerName="test-curl-7784-455d3a"
Sep 10 13:41:33.590188 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:33.590102    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e3e9bab-0125-42e8-8c1f-57094cde86e2" containerName="test-curl-7784-455d3a"
Sep 10 13:41:33.590243 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:33.590202    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e3e9bab-0125-42e8-8c1f-57094cde86e2" containerName="test-curl-7784-455d3a"
Sep 10 13:41:33.593297 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:33.593276    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-398268"
Sep 10 13:41:33.601605 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:33.601568    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-398268"]
Sep 10 13:41:33.707391 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:33.707364    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtw74\" (UniqueName: \"kubernetes.io/projected/95278f8f-7c6d-46af-a5f3-5603a598d8f5-kube-api-access-dtw74\") pod \"test-curl-7784-398268\" (UID: \"95278f8f-7c6d-46af-a5f3-5603a598d8f5\") " pod="opendatahub/test-curl-7784-398268"
Sep 10 13:41:33.778726 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:41:33.778706    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7784-455d3a" container="test-curl-7784-455d3a"
Sep 10 13:41:33.808865 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:33.808837    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtw74\" (UniqueName: \"kubernetes.io/projected/95278f8f-7c6d-46af-a5f3-5603a598d8f5-kube-api-access-dtw74\") pod \"test-curl-7784-398268\" (UID: \"95278f8f-7c6d-46af-a5f3-5603a598d8f5\") " pod="opendatahub/test-curl-7784-398268"
Sep 10 13:41:33.818473 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:33.818455    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtw74\" (UniqueName: \"kubernetes.io/projected/95278f8f-7c6d-46af-a5f3-5603a598d8f5-kube-api-access-dtw74\") pod \"test-curl-7784-398268\" (UID: \"95278f8f-7c6d-46af-a5f3-5603a598d8f5\") " pod="opendatahub/test-curl-7784-398268"
Sep 10 13:41:33.902296 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:33.902274    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-398268"
Sep 10 13:41:34.181603 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:34.181565    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-398268"]
Sep 10 13:41:34.183267 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:41:34.183232    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95278f8f_7c6d_46af_a5f3_5603a598d8f5.slice/crio-bc4945e1d78530e5ad7aed33b385c8a3150b455472beec59b8ef4b02fa492f38 WatchSource:0}: Error finding container bc4945e1d78530e5ad7aed33b385c8a3150b455472beec59b8ef4b02fa492f38: Status 404 returned error can't find the container with id bc4945e1d78530e5ad7aed33b385c8a3150b455472beec59b8ef4b02fa492f38
Sep 10 13:41:34.785379 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:34.785345    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-398268" event={"ID":"95278f8f-7c6d-46af-a5f3-5603a598d8f5","Type":"ContainerStarted","Data":"7b4473dc374569e8c61f9a0c804a83abda27cc8d9b660dd9b055e6ed1e4ab165"}
Sep 10 13:41:34.785827 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:34.785389    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-398268" event={"ID":"95278f8f-7c6d-46af-a5f3-5603a598d8f5","Type":"ContainerStarted","Data":"bc4945e1d78530e5ad7aed33b385c8a3150b455472beec59b8ef4b02fa492f38"}
Sep 10 13:41:34.785827 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:34.785452    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7784-455d3a" podUID="4e3e9bab-0125-42e8-8c1f-57094cde86e2" containerName="test-curl-7784-455d3a" containerID="cri-o://b3b991f1e3d0d3e4e7508bbe1af90bff434354aec84737ef1f2df74de6fe65b4" gracePeriod=2
Sep 10 13:41:34.791028 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:34.791008    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:41:34.806284 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:34.806246    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7784-398268" podStartSLOduration=1.806233779 podStartE2EDuration="1.806233779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:41:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:41:34.805673679 +0000 UTC m=+1405.624712079" watchObservedRunningTime="2026-09-10 13:41:34.806233779 +0000 UTC m=+1405.625272178"
Sep 10 13:41:35.581362 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:35.581332    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7784-398268"]
Sep 10 13:41:35.587776 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:35.587752    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7784-398268"]
Sep 10 13:41:35.699164 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:35.699095    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7784-f111bd"]
Sep 10 13:41:35.699455 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:35.699442    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95278f8f-7c6d-46af-a5f3-5603a598d8f5" containerName="test-curl-7784-398268"
Sep 10 13:41:35.699501 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:35.699456    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="95278f8f-7c6d-46af-a5f3-5603a598d8f5" containerName="test-curl-7784-398268"
Sep 10 13:41:35.699544 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:35.699539    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="95278f8f-7c6d-46af-a5f3-5603a598d8f5" containerName="test-curl-7784-398268"
Sep 10 13:41:35.702637 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:35.702619    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-f111bd"
Sep 10 13:41:35.715807 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:35.715785    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-f111bd"]
Sep 10 13:41:35.790342 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:41:35.790313    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7784-398268" container="test-curl-7784-398268"
Sep 10 13:41:35.827068 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:35.827027    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttgdz\" (UniqueName: \"kubernetes.io/projected/3bff4e63-a399-459b-bf10-77a2243aeb7d-kube-api-access-ttgdz\") pod \"test-curl-7784-f111bd\" (UID: \"3bff4e63-a399-459b-bf10-77a2243aeb7d\") " pod="opendatahub/test-curl-7784-f111bd"
Sep 10 13:41:35.928503 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:35.928466    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttgdz\" (UniqueName: \"kubernetes.io/projected/3bff4e63-a399-459b-bf10-77a2243aeb7d-kube-api-access-ttgdz\") pod \"test-curl-7784-f111bd\" (UID: \"3bff4e63-a399-459b-bf10-77a2243aeb7d\") " pod="opendatahub/test-curl-7784-f111bd"
Sep 10 13:41:35.939712 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:35.939680    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttgdz\" (UniqueName: \"kubernetes.io/projected/3bff4e63-a399-459b-bf10-77a2243aeb7d-kube-api-access-ttgdz\") pod \"test-curl-7784-f111bd\" (UID: \"3bff4e63-a399-459b-bf10-77a2243aeb7d\") " pod="opendatahub/test-curl-7784-f111bd"
Sep 10 13:41:36.015810 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:36.015780    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-f111bd"
Sep 10 13:41:36.287610 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:36.287584    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-f111bd"]
Sep 10 13:41:36.289785 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:41:36.289757    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3bff4e63_a399_459b_bf10_77a2243aeb7d.slice/crio-af3ce8acc9418a7bbc0f4cbc0cfb3138fedd17d5066c279bfa47eaf961fdc9cc WatchSource:0}: Error finding container af3ce8acc9418a7bbc0f4cbc0cfb3138fedd17d5066c279bfa47eaf961fdc9cc: Status 404 returned error can't find the container with id af3ce8acc9418a7bbc0f4cbc0cfb3138fedd17d5066c279bfa47eaf961fdc9cc
Sep 10 13:41:36.797699 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:36.797665    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-f111bd" event={"ID":"3bff4e63-a399-459b-bf10-77a2243aeb7d","Type":"ContainerStarted","Data":"01890cfb817a04e63e25e847fdb2521cc37ba615bb05002223065169479b5663"}
Sep 10 13:41:36.798035 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:36.797721    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-f111bd" event={"ID":"3bff4e63-a399-459b-bf10-77a2243aeb7d","Type":"ContainerStarted","Data":"af3ce8acc9418a7bbc0f4cbc0cfb3138fedd17d5066c279bfa47eaf961fdc9cc"}
Sep 10 13:41:36.798035 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:36.797755    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7784-398268" podUID="95278f8f-7c6d-46af-a5f3-5603a598d8f5" containerName="test-curl-7784-398268" containerID="cri-o://7b4473dc374569e8c61f9a0c804a83abda27cc8d9b660dd9b055e6ed1e4ab165" gracePeriod=2
Sep 10 13:41:36.817780 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:36.817738    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7784-f111bd" podStartSLOduration=1.817722652 podStartE2EDuration="1.817722652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:41:36.816008544 +0000 UTC m=+1407.635046945" watchObservedRunningTime="2026-09-10 13:41:36.817722652 +0000 UTC m=+1407.636761077"
Sep 10 13:41:36.917045 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:36.917021    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-455d3a"
Sep 10 13:41:37.039623 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.039546    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx7w2\" (UniqueName: \"kubernetes.io/projected/4e3e9bab-0125-42e8-8c1f-57094cde86e2-kube-api-access-bx7w2\") pod \"4e3e9bab-0125-42e8-8c1f-57094cde86e2\" (UID: \"4e3e9bab-0125-42e8-8c1f-57094cde86e2\") "
Sep 10 13:41:37.041640 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.041606    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e3e9bab-0125-42e8-8c1f-57094cde86e2-kube-api-access-bx7w2" (OuterVolumeSpecName: "kube-api-access-bx7w2") pod "4e3e9bab-0125-42e8-8c1f-57094cde86e2" (UID: "4e3e9bab-0125-42e8-8c1f-57094cde86e2"). InnerVolumeSpecName "kube-api-access-bx7w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:41:37.140459 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.140426    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx7w2\" (UniqueName: \"kubernetes.io/projected/4e3e9bab-0125-42e8-8c1f-57094cde86e2-kube-api-access-bx7w2\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:41:37.480525 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.480492    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7784-f111bd"]
Sep 10 13:41:37.493759 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.493732    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7784-f111bd"]
Sep 10 13:41:37.611595 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.611540    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7784-f35153"]
Sep 10 13:41:37.611976 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.611959    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bff4e63-a399-459b-bf10-77a2243aeb7d" containerName="test-curl-7784-f111bd"
Sep 10 13:41:37.612024 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.611979    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bff4e63-a399-459b-bf10-77a2243aeb7d" containerName="test-curl-7784-f111bd"
Sep 10 13:41:37.612060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.612043    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bff4e63-a399-459b-bf10-77a2243aeb7d" containerName="test-curl-7784-f111bd"
Sep 10 13:41:37.615205 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.615189    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-f35153"
Sep 10 13:41:37.620068 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.620043    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-f35153"]
Sep 10 13:41:37.710402 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.710370    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e3e9bab-0125-42e8-8c1f-57094cde86e2" path="/var/lib/kubelet/pods/4e3e9bab-0125-42e8-8c1f-57094cde86e2/volumes"
Sep 10 13:41:37.744958 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.744888    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6lwj\" (UniqueName: \"kubernetes.io/projected/4edaab2b-db87-4826-80e1-e8fe661cfea9-kube-api-access-q6lwj\") pod \"test-curl-7784-f35153\" (UID: \"4edaab2b-db87-4826-80e1-e8fe661cfea9\") " pod="opendatahub/test-curl-7784-f35153"
Sep 10 13:41:37.803505 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.803471    2551 generic.go:356] "Generic (PLEG): container finished" podID="4e3e9bab-0125-42e8-8c1f-57094cde86e2" containerID="b3b991f1e3d0d3e4e7508bbe1af90bff434354aec84737ef1f2df74de6fe65b4" exitCode=137
Sep 10 13:41:37.803897 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.803515    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-455d3a"
Sep 10 13:41:37.803897 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.803566    2551 scope.go:117] "RemoveContainer" containerID="b3b991f1e3d0d3e4e7508bbe1af90bff434354aec84737ef1f2df74de6fe65b4"
Sep 10 13:41:37.803897 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:41:37.803712    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7784-f111bd" container="test-curl-7784-f111bd"
Sep 10 13:41:37.811995 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.811974    2551 scope.go:117] "RemoveContainer" containerID="b3b991f1e3d0d3e4e7508bbe1af90bff434354aec84737ef1f2df74de6fe65b4"
Sep 10 13:41:37.812291 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:41:37.812270    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3b991f1e3d0d3e4e7508bbe1af90bff434354aec84737ef1f2df74de6fe65b4\": container with ID starting with b3b991f1e3d0d3e4e7508bbe1af90bff434354aec84737ef1f2df74de6fe65b4 not found: ID does not exist" containerID="b3b991f1e3d0d3e4e7508bbe1af90bff434354aec84737ef1f2df74de6fe65b4"
Sep 10 13:41:37.812354 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.812300    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3b991f1e3d0d3e4e7508bbe1af90bff434354aec84737ef1f2df74de6fe65b4"} err="failed to get container status \"b3b991f1e3d0d3e4e7508bbe1af90bff434354aec84737ef1f2df74de6fe65b4\": rpc error: code = NotFound desc = could not find container \"b3b991f1e3d0d3e4e7508bbe1af90bff434354aec84737ef1f2df74de6fe65b4\": container with ID starting with b3b991f1e3d0d3e4e7508bbe1af90bff434354aec84737ef1f2df74de6fe65b4 not found: ID does not exist"
Sep 10 13:41:37.846182 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.846161    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6lwj\" (UniqueName: \"kubernetes.io/projected/4edaab2b-db87-4826-80e1-e8fe661cfea9-kube-api-access-q6lwj\") pod \"test-curl-7784-f35153\" (UID: \"4edaab2b-db87-4826-80e1-e8fe661cfea9\") " pod="opendatahub/test-curl-7784-f35153"
Sep 10 13:41:37.855521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.855494    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6lwj\" (UniqueName: \"kubernetes.io/projected/4edaab2b-db87-4826-80e1-e8fe661cfea9-kube-api-access-q6lwj\") pod \"test-curl-7784-f35153\" (UID: \"4edaab2b-db87-4826-80e1-e8fe661cfea9\") " pod="opendatahub/test-curl-7784-f35153"
Sep 10 13:41:37.925362 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:37.925336    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-f35153"
Sep 10 13:41:38.195772 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:38.195747    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7784-f35153"]
Sep 10 13:41:38.197876 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:41:38.197837    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4edaab2b_db87_4826_80e1_e8fe661cfea9.slice/crio-1da1b646b4bf415fbd9c4c2b36f824dddc79f11bdaff2ab904b2b68ca9ab1888 WatchSource:0}: Error finding container 1da1b646b4bf415fbd9c4c2b36f824dddc79f11bdaff2ab904b2b68ca9ab1888: Status 404 returned error can't find the container with id 1da1b646b4bf415fbd9c4c2b36f824dddc79f11bdaff2ab904b2b68ca9ab1888
Sep 10 13:41:38.811647 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:38.811616    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-f35153" event={"ID":"4edaab2b-db87-4826-80e1-e8fe661cfea9","Type":"ContainerStarted","Data":"1ade39c8c49e7a9e54dca11d252d0bf9901399d636a20f62a790ea88c7780e8e"}
Sep 10 13:41:38.812027 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:38.811665    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7784-f35153" event={"ID":"4edaab2b-db87-4826-80e1-e8fe661cfea9","Type":"ContainerStarted","Data":"1da1b646b4bf415fbd9c4c2b36f824dddc79f11bdaff2ab904b2b68ca9ab1888"}
Sep 10 13:41:38.813013 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:38.812982    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7784-f111bd" podUID="3bff4e63-a399-459b-bf10-77a2243aeb7d" containerName="test-curl-7784-f111bd" containerID="cri-o://01890cfb817a04e63e25e847fdb2521cc37ba615bb05002223065169479b5663" gracePeriod=2
Sep 10 13:41:38.830633 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:38.830569    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7784-f35153" podStartSLOduration=1.830551397 podStartE2EDuration="1.830551397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:41:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:41:38.83005578 +0000 UTC m=+1409.649094181" watchObservedRunningTime="2026-09-10 13:41:38.830551397 +0000 UTC m=+1409.649589799"
Sep 10 13:41:38.934166 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:38.934144    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-398268"
Sep 10 13:41:39.057221 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:39.057141    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtw74\" (UniqueName: \"kubernetes.io/projected/95278f8f-7c6d-46af-a5f3-5603a598d8f5-kube-api-access-dtw74\") pod \"95278f8f-7c6d-46af-a5f3-5603a598d8f5\" (UID: \"95278f8f-7c6d-46af-a5f3-5603a598d8f5\") "
Sep 10 13:41:39.059159 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:39.059134    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95278f8f-7c6d-46af-a5f3-5603a598d8f5-kube-api-access-dtw74" (OuterVolumeSpecName: "kube-api-access-dtw74") pod "95278f8f-7c6d-46af-a5f3-5603a598d8f5" (UID: "95278f8f-7c6d-46af-a5f3-5603a598d8f5"). InnerVolumeSpecName "kube-api-access-dtw74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:41:39.158037 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:39.158005    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtw74\" (UniqueName: \"kubernetes.io/projected/95278f8f-7c6d-46af-a5f3-5603a598d8f5-kube-api-access-dtw74\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:41:39.495595 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:39.495555    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7784-f35153"]
Sep 10 13:41:39.504951 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:39.504920    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7784-f35153"]
Sep 10 13:41:39.717272 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:39.717235    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95278f8f-7c6d-46af-a5f3-5603a598d8f5" path="/var/lib/kubelet/pods/95278f8f-7c6d-46af-a5f3-5603a598d8f5/volumes"
Sep 10 13:41:39.819095 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:39.819010    2551 generic.go:356] "Generic (PLEG): container finished" podID="95278f8f-7c6d-46af-a5f3-5603a598d8f5" containerID="7b4473dc374569e8c61f9a0c804a83abda27cc8d9b660dd9b055e6ed1e4ab165" exitCode=137
Sep 10 13:41:39.819095 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:39.819062    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-398268"
Sep 10 13:41:39.819570 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:39.819150    2551 scope.go:117] "RemoveContainer" containerID="7b4473dc374569e8c61f9a0c804a83abda27cc8d9b660dd9b055e6ed1e4ab165"
Sep 10 13:41:39.819570 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:41:39.819252    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7784-f35153" container="test-curl-7784-f35153"
Sep 10 13:41:39.827721 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:39.827701    2551 scope.go:117] "RemoveContainer" containerID="7b4473dc374569e8c61f9a0c804a83abda27cc8d9b660dd9b055e6ed1e4ab165"
Sep 10 13:41:39.827994 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:41:39.827974    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b4473dc374569e8c61f9a0c804a83abda27cc8d9b660dd9b055e6ed1e4ab165\": container with ID starting with 7b4473dc374569e8c61f9a0c804a83abda27cc8d9b660dd9b055e6ed1e4ab165 not found: ID does not exist" containerID="7b4473dc374569e8c61f9a0c804a83abda27cc8d9b660dd9b055e6ed1e4ab165"
Sep 10 13:41:39.828061 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:39.828007    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b4473dc374569e8c61f9a0c804a83abda27cc8d9b660dd9b055e6ed1e4ab165"} err="failed to get container status \"7b4473dc374569e8c61f9a0c804a83abda27cc8d9b660dd9b055e6ed1e4ab165\": rpc error: code = NotFound desc = could not find container \"7b4473dc374569e8c61f9a0c804a83abda27cc8d9b660dd9b055e6ed1e4ab165\": container with ID starting with 7b4473dc374569e8c61f9a0c804a83abda27cc8d9b660dd9b055e6ed1e4ab165 not found: ID does not exist"
Sep 10 13:41:40.403255 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.403212    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"]
Sep 10 13:41:40.403855 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.403835    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4edaab2b-db87-4826-80e1-e8fe661cfea9" containerName="test-curl-7784-f35153"
Sep 10 13:41:40.403915 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.403859    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="4edaab2b-db87-4826-80e1-e8fe661cfea9" containerName="test-curl-7784-f35153"
Sep 10 13:41:40.403962 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.403946    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="4edaab2b-db87-4826-80e1-e8fe661cfea9" containerName="test-curl-7784-f35153"
Sep 10 13:41:40.408480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.408461    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.411121 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.411099    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9469c838-openshift-default-dockercfg-mz5d4\""
Sep 10 13:41:40.421257 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.421233    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"]
Sep 10 13:41:40.571326 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.571281    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-workload-certs\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.571506 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.571344    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-token\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.571506 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.571416    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-envoy\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.571506 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.571451    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-podinfo\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.571506 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.571476    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmjd5\" (UniqueName: \"kubernetes.io/projected/3c28acdd-f70d-4f63-8376-50fe7398c58d-kube-api-access-hmjd5\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.571506 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.571499    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-credential-socket\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.571742 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.571558    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c28acdd-f70d-4f63-8376-50fe7398c58d-istiod-ca-cert\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.571742 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.571675    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-data\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.571742 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.571713    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-workload-socket\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.672753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.672675    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-workload-certs\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.672753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.672715    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-token\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.672966 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.672758    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-envoy\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.672966 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.672802    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-podinfo\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.672966 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.672834    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmjd5\" (UniqueName: \"kubernetes.io/projected/3c28acdd-f70d-4f63-8376-50fe7398c58d-kube-api-access-hmjd5\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.672966 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.672861    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-credential-socket\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.672966 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.672889    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c28acdd-f70d-4f63-8376-50fe7398c58d-istiod-ca-cert\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.672966 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.672932    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-data\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.673276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.672993    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-workload-socket\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.673276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.673113    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-workload-certs\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.673276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.673218    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-credential-socket\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.673415 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.673369    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-workload-socket\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.673415 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.673396    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-data\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.673531 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.673513    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c28acdd-f70d-4f63-8376-50fe7398c58d-istiod-ca-cert\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.675560 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.675534    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-envoy\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.675894 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.675871    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-podinfo\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.694713 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.694684    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmjd5\" (UniqueName: \"kubernetes.io/projected/3c28acdd-f70d-4f63-8376-50fe7398c58d-kube-api-access-hmjd5\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.694967 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.694950    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-token\") pod \"e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") " pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.718162 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.718136    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:40.826369 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.826340    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7784-f35153" podUID="4edaab2b-db87-4826-80e1-e8fe661cfea9" containerName="test-curl-7784-f35153" containerID="cri-o://1ade39c8c49e7a9e54dca11d252d0bf9901399d636a20f62a790ea88c7780e8e" gracePeriod=2
Sep 10 13:41:40.964165 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:40.964139    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-f111bd"
Sep 10 13:41:41.005672 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.005643    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"]
Sep 10 13:41:41.007509 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:41:41.007484    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c28acdd_f70d_4f63_8376_50fe7398c58d.slice/crio-a674656a1333eabcfe1ae7b7b93e6a5752a6a31cf07a5b1190748874d2cbad20 WatchSource:0}: Error finding container a674656a1333eabcfe1ae7b7b93e6a5752a6a31cf07a5b1190748874d2cbad20: Status 404 returned error can't find the container with id a674656a1333eabcfe1ae7b7b93e6a5752a6a31cf07a5b1190748874d2cbad20
Sep 10 13:41:41.009830 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.009797    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:41:41.009915 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.009873    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:41:41.009985 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.009919    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:41:41.077269 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.077222    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttgdz\" (UniqueName: \"kubernetes.io/projected/3bff4e63-a399-459b-bf10-77a2243aeb7d-kube-api-access-ttgdz\") pod \"3bff4e63-a399-459b-bf10-77a2243aeb7d\" (UID: \"3bff4e63-a399-459b-bf10-77a2243aeb7d\") "
Sep 10 13:41:41.080012 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.079919    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bff4e63-a399-459b-bf10-77a2243aeb7d-kube-api-access-ttgdz" (OuterVolumeSpecName: "kube-api-access-ttgdz") pod "3bff4e63-a399-459b-bf10-77a2243aeb7d" (UID: "3bff4e63-a399-459b-bf10-77a2243aeb7d"). InnerVolumeSpecName "kube-api-access-ttgdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:41:41.179330 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.179290    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttgdz\" (UniqueName: \"kubernetes.io/projected/3bff4e63-a399-459b-bf10-77a2243aeb7d-kube-api-access-ttgdz\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:41:41.710219 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.710187    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bff4e63-a399-459b-bf10-77a2243aeb7d" path="/var/lib/kubelet/pods/3bff4e63-a399-459b-bf10-77a2243aeb7d/volumes"
Sep 10 13:41:41.835017 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.834976    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h" event={"ID":"3c28acdd-f70d-4f63-8376-50fe7398c58d","Type":"ContainerStarted","Data":"61f4477a757c86c7be3a9b912a76bfbe4af743cd71ceb0443cb4efcbd05711f4"}
Sep 10 13:41:41.835549 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.835031    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h" event={"ID":"3c28acdd-f70d-4f63-8376-50fe7398c58d","Type":"ContainerStarted","Data":"a674656a1333eabcfe1ae7b7b93e6a5752a6a31cf07a5b1190748874d2cbad20"}
Sep 10 13:41:41.836266 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.836241    2551 generic.go:356] "Generic (PLEG): container finished" podID="3bff4e63-a399-459b-bf10-77a2243aeb7d" containerID="01890cfb817a04e63e25e847fdb2521cc37ba615bb05002223065169479b5663" exitCode=137
Sep 10 13:41:41.836359 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.836276    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-f111bd"
Sep 10 13:41:41.836359 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.836287    2551 scope.go:117] "RemoveContainer" containerID="01890cfb817a04e63e25e847fdb2521cc37ba615bb05002223065169479b5663"
Sep 10 13:41:41.844691 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.844668    2551 scope.go:117] "RemoveContainer" containerID="01890cfb817a04e63e25e847fdb2521cc37ba615bb05002223065169479b5663"
Sep 10 13:41:41.845010 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:41:41.844984    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01890cfb817a04e63e25e847fdb2521cc37ba615bb05002223065169479b5663\": container with ID starting with 01890cfb817a04e63e25e847fdb2521cc37ba615bb05002223065169479b5663 not found: ID does not exist" containerID="01890cfb817a04e63e25e847fdb2521cc37ba615bb05002223065169479b5663"
Sep 10 13:41:41.845061 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.845012    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01890cfb817a04e63e25e847fdb2521cc37ba615bb05002223065169479b5663"} err="failed to get container status \"01890cfb817a04e63e25e847fdb2521cc37ba615bb05002223065169479b5663\": rpc error: code = NotFound desc = could not find container \"01890cfb817a04e63e25e847fdb2521cc37ba615bb05002223065169479b5663\": container with ID starting with 01890cfb817a04e63e25e847fdb2521cc37ba615bb05002223065169479b5663 not found: ID does not exist"
Sep 10 13:41:41.869652 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:41.869606    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h" podStartSLOduration=1.8695870810000002 podStartE2EDuration="1.869587081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:41:41.868733197 +0000 UTC m=+1412.687771598" watchObservedRunningTime="2026-09-10 13:41:41.869587081 +0000 UTC m=+1412.688625485"
Sep 10 13:41:42.718468 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:42.718423    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:42.723967 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:42.723939    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:42.844006 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:42.843981    2551 generic.go:356] "Generic (PLEG): container finished" podID="4edaab2b-db87-4826-80e1-e8fe661cfea9" containerID="1ade39c8c49e7a9e54dca11d252d0bf9901399d636a20f62a790ea88c7780e8e" exitCode=137
Sep 10 13:41:42.845443 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:42.845420    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:42.846340 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:42.846320    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:41:42.973643 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:42.971693    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-f35153"
Sep 10 13:41:42.998625 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:42.998590    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6lwj\" (UniqueName: \"kubernetes.io/projected/4edaab2b-db87-4826-80e1-e8fe661cfea9-kube-api-access-q6lwj\") pod \"4edaab2b-db87-4826-80e1-e8fe661cfea9\" (UID: \"4edaab2b-db87-4826-80e1-e8fe661cfea9\") "
Sep 10 13:41:43.000603 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:43.000559    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4edaab2b-db87-4826-80e1-e8fe661cfea9-kube-api-access-q6lwj" (OuterVolumeSpecName: "kube-api-access-q6lwj") pod "4edaab2b-db87-4826-80e1-e8fe661cfea9" (UID: "4edaab2b-db87-4826-80e1-e8fe661cfea9"). InnerVolumeSpecName "kube-api-access-q6lwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:41:43.099491 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:43.099457    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6lwj\" (UniqueName: \"kubernetes.io/projected/4edaab2b-db87-4826-80e1-e8fe661cfea9-kube-api-access-q6lwj\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:41:43.710078 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:43.710040    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4edaab2b-db87-4826-80e1-e8fe661cfea9" path="/var/lib/kubelet/pods/4edaab2b-db87-4826-80e1-e8fe661cfea9/volumes"
Sep 10 13:41:43.851439 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:43.851414    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7784-f35153"
Sep 10 13:41:43.851872 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:43.851472    2551 scope.go:117] "RemoveContainer" containerID="1ade39c8c49e7a9e54dca11d252d0bf9901399d636a20f62a790ea88c7780e8e"
Sep 10 13:41:53.477477 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:41:53.477446    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:42:03.957834 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:03.957800    2551 generic.go:356] "Generic (PLEG): container finished" podID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerID="61f4477a757c86c7be3a9b912a76bfbe4af743cd71ceb0443cb4efcbd05711f4" exitCode=0
Sep 10 13:42:03.958262 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:03.957877    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h" event={"ID":"3c28acdd-f70d-4f63-8376-50fe7398c58d","Type":"ContainerDied","Data":"61f4477a757c86c7be3a9b912a76bfbe4af743cd71ceb0443cb4efcbd05711f4"}
Sep 10 13:42:03.958262 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:03.958258    2551 scope.go:117] "RemoveContainer" containerID="61f4477a757c86c7be3a9b912a76bfbe4af743cd71ceb0443cb4efcbd05711f4"
Sep 10 13:42:03.959294 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:03.959267    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:42:03.959348 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:03.959335    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:42:03.959393 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:03.959369    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:42:04.968112 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:04.968072    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h" event={"ID":"3c28acdd-f70d-4f63-8376-50fe7398c58d","Type":"ContainerStarted","Data":"dc9e31709917f92a19454920dd7898fa36595923b18b75d7d8eb5ace92c27652"}
Sep 10 13:42:05.718335 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:05.718305    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:42:05.718543 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:05.718516    2551 patch_prober.go:28] interesting pod/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 10 13:42:05.718664 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:05.718569    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h" podUID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 10 13:42:06.719377 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:06.719343    2551 patch_prober.go:28] interesting pod/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 10 13:42:06.719830 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:06.719403    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h" podUID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 10 13:42:07.719139 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:07.719108    2551 patch_prober.go:28] interesting pod/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 10 13:42:07.719310 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:07.719156    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h" podUID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 10 13:42:08.719338 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:08.719304    2551 patch_prober.go:28] interesting pod/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 10 13:42:08.719769 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:08.719364    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h" podUID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 10 13:42:09.719045 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:09.719014    2551 patch_prober.go:28] interesting pod/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 10 13:42:09.719206 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:09.719060    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h" podUID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 10 13:42:09.783444 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:09.783412    2551 scope.go:117] "RemoveContainer" containerID="bcc6077a13acdc6ecde7daed89d47ef3df54cf54bac1a11ce8c5f037d16b3aa0"
Sep 10 13:42:09.791935 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:09.791914    2551 scope.go:117] "RemoveContainer" containerID="42008e199cb3b74a1a4c7d7ed2ba9d9c9a497173d1bb12fd9118deb40c3fc844"
Sep 10 13:42:10.722620 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:10.722592    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:42:10.722978 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:10.722958    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:42:10.723640 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:10.723618    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:42:10.891511 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:10.891486    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:42:14.520673 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.520639    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"]
Sep 10 13:42:14.525661 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.525640    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.530089 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.530068    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9469c838\"/\"kube-root-ca.crt\""
Sep 10 13:42:14.531010 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.530990    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9469c838\"/\"default-dockercfg-gww8v\""
Sep 10 13:42:14.531107 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.531029    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9469c838\"/\"e2e-lifecycle-model-9469c838-kserve-self-signed-certs\""
Sep 10 13:42:14.537653 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.537634    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9469c838\"/\"openshift-service-ca.crt\""
Sep 10 13:42:14.547207 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.547184    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"]
Sep 10 13:42:14.622207 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.622178    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-home\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.622360 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.622212    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-tmp-dir\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.622360 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.622307    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-model-cache\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.622360 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.622347    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e87d0aa1-0827-4945-900d-79f42a611524-tls-certs\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.622472 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.622373    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrlng\" (UniqueName: \"kubernetes.io/projected/e87d0aa1-0827-4945-900d-79f42a611524-kube-api-access-hrlng\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.622507 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.622477    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-dshm\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.723650 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.723617    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-dshm\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.723840 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.723672    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-home\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.723840 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.723696    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-tmp-dir\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.723840 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.723796    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-model-cache\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.724015 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.723850    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e87d0aa1-0827-4945-900d-79f42a611524-tls-certs\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.724015 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.723892    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrlng\" (UniqueName: \"kubernetes.io/projected/e87d0aa1-0827-4945-900d-79f42a611524-kube-api-access-hrlng\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.724125 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.724101    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-home\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.724125 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.724116    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-tmp-dir\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.724237 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.724155    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-model-cache\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.725889 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.725866    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-dshm\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.726166 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.726148    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e87d0aa1-0827-4945-900d-79f42a611524-tls-certs\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.734646 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.734625    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrlng\" (UniqueName: \"kubernetes.io/projected/e87d0aa1-0827-4945-900d-79f42a611524-kube-api-access-hrlng\") pod \"e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") " pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:14.834047 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:14.833977    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:15.105931 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:15.105906    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"]
Sep 10 13:42:15.108104 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:42:15.108078    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode87d0aa1_0827_4945_900d_79f42a611524.slice/crio-691493bba287ed1c64d9f454692e644110c12a284dba2bdf1695907d4f08a22c WatchSource:0}: Error finding container 691493bba287ed1c64d9f454692e644110c12a284dba2bdf1695907d4f08a22c: Status 404 returned error can't find the container with id 691493bba287ed1c64d9f454692e644110c12a284dba2bdf1695907d4f08a22c
Sep 10 13:42:16.029429 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:16.029393    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx" event={"ID":"e87d0aa1-0827-4945-900d-79f42a611524","Type":"ContainerStarted","Data":"3cb05b83821e3394a724c8850d909c694b7c1ba23ca795cecd8f175604f6f543"}
Sep 10 13:42:16.029827 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:16.029437    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx" event={"ID":"e87d0aa1-0827-4945-900d-79f42a611524","Type":"ContainerStarted","Data":"691493bba287ed1c64d9f454692e644110c12a284dba2bdf1695907d4f08a22c"}
Sep 10 13:42:16.047558 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:16.047500    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx" podStartSLOduration=2.047486056 podStartE2EDuration="2.047486056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:42:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:42:16.04644971 +0000 UTC m=+1446.865488123" watchObservedRunningTime="2026-09-10 13:42:16.047486056 +0000 UTC m=+1446.866524456"
Sep 10 13:42:24.834432 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:24.834382    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:24.834892 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:24.834446    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:24.847558 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:24.847533    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:25.086924 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:25.086844    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:42:28.785919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:28.785887    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:42:43.466753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:43.466708    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz"]
Sep 10 13:42:43.470420 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:43.470396    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz"
Sep 10 13:42:43.473079 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:43.473055    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:42:43.473196 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:43.473060    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:42:43.479346 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:43.479316    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz"]
Sep 10 13:42:43.619605 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:43.619521    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9469c838-h4hzz\" (UID: \"1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz"
Sep 10 13:42:43.720327 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:43.720233    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9469c838-h4hzz\" (UID: \"1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz"
Sep 10 13:42:43.720942 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:43.720917    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9469c838-h4hzz\" (UID: \"1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz"
Sep 10 13:42:43.783240 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:43.783202    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz"
Sep 10 13:42:44.055749 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:44.055718    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz"]
Sep 10 13:42:44.057012 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:42:44.056979    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd19536_f2cb_4bb9_91c2_3c1a3768b1ca.slice/crio-eebd5c74db65ed3489136f08c59cde6aa7cfa6790bd570c1b9f0bf4a3b23f413 WatchSource:0}: Error finding container eebd5c74db65ed3489136f08c59cde6aa7cfa6790bd570c1b9f0bf4a3b23f413: Status 404 returned error can't find the container with id eebd5c74db65ed3489136f08c59cde6aa7cfa6790bd570c1b9f0bf4a3b23f413
Sep 10 13:42:44.173066 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:44.173027    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz" event={"ID":"1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca","Type":"ContainerStarted","Data":"eebd5c74db65ed3489136f08c59cde6aa7cfa6790bd570c1b9f0bf4a3b23f413"}
Sep 10 13:42:45.180509 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:45.180421    2551 generic.go:356] "Generic (PLEG): container finished" podID="1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca" containerID="fa37a3fe1eb70dbbd6db975fa09252098d9c02edf87a63a4a8e81e15364101a1" exitCode=0
Sep 10 13:42:45.180509 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:45.180469    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz" event={"ID":"1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca","Type":"ContainerDied","Data":"fa37a3fe1eb70dbbd6db975fa09252098d9c02edf87a63a4a8e81e15364101a1"}
Sep 10 13:42:46.311806 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:46.311778    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz"
Sep 10 13:42:46.450981 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:46.450893    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca-maas-api-service-ca\") pod \"1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca\" (UID: \"1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca\") "
Sep 10 13:42:46.451317 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:46.451287    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca" (UID: "1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:42:46.552442 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:46.552411    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:42:47.192049 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:47.192013    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz"
Sep 10 13:42:47.192049 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:47.192038    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz" event={"ID":"1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca","Type":"ContainerDied","Data":"eebd5c74db65ed3489136f08c59cde6aa7cfa6790bd570c1b9f0bf4a3b23f413"}
Sep 10 13:42:47.192249 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:47.192087    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eebd5c74db65ed3489136f08c59cde6aa7cfa6790bd570c1b9f0bf4a3b23f413"
Sep 10 13:42:47.770381 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:47.770349    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:42:59.379362 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:59.379322    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz"]
Sep 10 13:42:59.384173 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:59.384144    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9469c838-h4hzz"]
Sep 10 13:42:59.709900 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:42:59.709825    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca" path="/var/lib/kubelet/pods/1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca/volumes"
Sep 10 13:43:06.271343 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:06.271302    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:43:07.529129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:07.529094    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"]
Sep 10 13:43:09.370148 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.370114    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7787-bf82e8"]
Sep 10 13:43:09.370505 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.370476    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca" containerName="revoke-keys"
Sep 10 13:43:09.370505 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.370488    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca" containerName="revoke-keys"
Sep 10 13:43:09.370602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.370548    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cd19536-f2cb-4bb9-91c2-3c1a3768b1ca" containerName="revoke-keys"
Sep 10 13:43:09.374833 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.374809    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7787-bf82e8"
Sep 10 13:43:09.377865 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.377845    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lpqwx\""
Sep 10 13:43:09.380411 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.380386    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7787-bf82e8"]
Sep 10 13:43:09.486887 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.486857    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbz8v\" (UniqueName: \"kubernetes.io/projected/6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3-kube-api-access-lbz8v\") pod \"test-curl-7787-bf82e8\" (UID: \"6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3\") " pod="opendatahub/test-curl-7787-bf82e8"
Sep 10 13:43:09.587766 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.587738    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbz8v\" (UniqueName: \"kubernetes.io/projected/6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3-kube-api-access-lbz8v\") pod \"test-curl-7787-bf82e8\" (UID: \"6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3\") " pod="opendatahub/test-curl-7787-bf82e8"
Sep 10 13:43:09.597326 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.597294    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbz8v\" (UniqueName: \"kubernetes.io/projected/6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3-kube-api-access-lbz8v\") pod \"test-curl-7787-bf82e8\" (UID: \"6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3\") " pod="opendatahub/test-curl-7787-bf82e8"
Sep 10 13:43:09.687876 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.687813    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lpqwx\""
Sep 10 13:43:09.695634 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.695615    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7787-bf82e8"
Sep 10 13:43:09.716344 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.716322    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:43:09.721667 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.721647    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:43:09.828176 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.828151    2551 scope.go:117] "RemoveContainer" containerID="201bb7e4c5baf74658d337b2611123586775ecbdb9d98f8c886c2f3ffa756193"
Sep 10 13:43:09.992331 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:09.992305    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7787-bf82e8"]
Sep 10 13:43:10.310685 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:10.310569    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7787-bf82e8" event={"ID":"6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3","Type":"ContainerStarted","Data":"8e427d85052da0290f456afb83f9c72b2251f4c47f9e78843f28dd201c3cf9eb"}
Sep 10 13:43:10.310685 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:10.310636    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7787-bf82e8" event={"ID":"6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3","Type":"ContainerStarted","Data":"5f7b528d9a92b866f2b95405f751bf157383c4745abd7f15e9ef660ac6bba46d"}
Sep 10 13:43:10.333978 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:10.333935    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7787-bf82e8" podStartSLOduration=1.333921742 podStartE2EDuration="1.333921742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:43:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:43:10.329454124 +0000 UTC m=+1501.148492521" watchObservedRunningTime="2026-09-10 13:43:10.333921742 +0000 UTC m=+1501.152960180"
Sep 10 13:43:10.993080 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:10.993052    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7787-bf82e8"]
Sep 10 13:43:10.997833 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:10.997806    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7787-bf82e8"]
Sep 10 13:43:11.315772 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:43:11.315745    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7787-bf82e8" container="test-curl-7787-bf82e8"
Sep 10 13:43:12.321358 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:12.321312    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7787-bf82e8" podUID="6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3" containerName="test-curl-7787-bf82e8" containerID="cri-o://8e427d85052da0290f456afb83f9c72b2251f4c47f9e78843f28dd201c3cf9eb" gracePeriod=2
Sep 10 13:43:12.395934 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:12.395898    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"]
Sep 10 13:43:13.624956 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:13.624921    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"]
Sep 10 13:43:14.511405 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:14.511376    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7787-bf82e8"
Sep 10 13:43:14.634687 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:14.634567    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbz8v\" (UniqueName: \"kubernetes.io/projected/6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3-kube-api-access-lbz8v\") pod \"6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3\" (UID: \"6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3\") "
Sep 10 13:43:14.636753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:14.636724    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3-kube-api-access-lbz8v" (OuterVolumeSpecName: "kube-api-access-lbz8v") pod "6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3" (UID: "6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3"). InnerVolumeSpecName "kube-api-access-lbz8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:43:14.735758 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:14.735720    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbz8v\" (UniqueName: \"kubernetes.io/projected/6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3-kube-api-access-lbz8v\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:15.340395 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:15.340357    2551 generic.go:356] "Generic (PLEG): container finished" podID="6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3" containerID="8e427d85052da0290f456afb83f9c72b2251f4c47f9e78843f28dd201c3cf9eb" exitCode=137
Sep 10 13:43:15.340619 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:15.340408    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7787-bf82e8"
Sep 10 13:43:15.340619 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:15.340454    2551 scope.go:117] "RemoveContainer" containerID="8e427d85052da0290f456afb83f9c72b2251f4c47f9e78843f28dd201c3cf9eb"
Sep 10 13:43:15.349320 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:15.349298    2551 scope.go:117] "RemoveContainer" containerID="8e427d85052da0290f456afb83f9c72b2251f4c47f9e78843f28dd201c3cf9eb"
Sep 10 13:43:15.349645 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:43:15.349624    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e427d85052da0290f456afb83f9c72b2251f4c47f9e78843f28dd201c3cf9eb\": container with ID starting with 8e427d85052da0290f456afb83f9c72b2251f4c47f9e78843f28dd201c3cf9eb not found: ID does not exist" containerID="8e427d85052da0290f456afb83f9c72b2251f4c47f9e78843f28dd201c3cf9eb"
Sep 10 13:43:15.349719 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:15.349652    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e427d85052da0290f456afb83f9c72b2251f4c47f9e78843f28dd201c3cf9eb"} err="failed to get container status \"8e427d85052da0290f456afb83f9c72b2251f4c47f9e78843f28dd201c3cf9eb\": rpc error: code = NotFound desc = could not find container \"8e427d85052da0290f456afb83f9c72b2251f4c47f9e78843f28dd201c3cf9eb\": container with ID starting with 8e427d85052da0290f456afb83f9c72b2251f4c47f9e78843f28dd201c3cf9eb not found: ID does not exist"
Sep 10 13:43:15.352743 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:43:15.352720    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7787-bf82e8\" is forbidden: User \"system:node:ip-10-0-135-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-117.ec2.internal' and this object" podUID="6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3" pod="opendatahub/test-curl-7787-bf82e8"
Sep 10 13:43:15.710446 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:15.710364    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3" path="/var/lib/kubelet/pods/6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3/volumes"
Sep 10 13:43:18.773022 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:18.772986    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:43:22.548796 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.548754    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx" podUID="e87d0aa1-0827-4945-900d-79f42a611524" containerName="main" containerID="cri-o://3cb05b83821e3394a724c8850d909c694b7c1ba23ca795cecd8f175604f6f543" gracePeriod=45
Sep 10 13:43:22.799782 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.799728    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:43:22.916283 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.916251    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrlng\" (UniqueName: \"kubernetes.io/projected/e87d0aa1-0827-4945-900d-79f42a611524-kube-api-access-hrlng\") pod \"e87d0aa1-0827-4945-900d-79f42a611524\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") "
Sep 10 13:43:22.916283 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.916289    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e87d0aa1-0827-4945-900d-79f42a611524-tls-certs\") pod \"e87d0aa1-0827-4945-900d-79f42a611524\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") "
Sep 10 13:43:22.916529 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.916378    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-home\") pod \"e87d0aa1-0827-4945-900d-79f42a611524\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") "
Sep 10 13:43:22.916529 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.916408    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-model-cache\") pod \"e87d0aa1-0827-4945-900d-79f42a611524\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") "
Sep 10 13:43:22.916529 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.916440    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-dshm\") pod \"e87d0aa1-0827-4945-900d-79f42a611524\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") "
Sep 10 13:43:22.916529 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.916485    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-tmp-dir\") pod \"e87d0aa1-0827-4945-900d-79f42a611524\" (UID: \"e87d0aa1-0827-4945-900d-79f42a611524\") "
Sep 10 13:43:22.916782 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.916677    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-home" (OuterVolumeSpecName: "home") pod "e87d0aa1-0827-4945-900d-79f42a611524" (UID: "e87d0aa1-0827-4945-900d-79f42a611524"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:22.916841 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.916770    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-model-cache" (OuterVolumeSpecName: "model-cache") pod "e87d0aa1-0827-4945-900d-79f42a611524" (UID: "e87d0aa1-0827-4945-900d-79f42a611524"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:22.916841 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.916782    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e87d0aa1-0827-4945-900d-79f42a611524" (UID: "e87d0aa1-0827-4945-900d-79f42a611524"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:22.916979 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.916957    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-home\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:22.916979 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.916980    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-model-cache\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:22.917138 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.916994    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-tmp-dir\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:22.918549 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.918520    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-dshm" (OuterVolumeSpecName: "dshm") pod "e87d0aa1-0827-4945-900d-79f42a611524" (UID: "e87d0aa1-0827-4945-900d-79f42a611524"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:22.918663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.918557    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e87d0aa1-0827-4945-900d-79f42a611524-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e87d0aa1-0827-4945-900d-79f42a611524" (UID: "e87d0aa1-0827-4945-900d-79f42a611524"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:43:22.918663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:22.918591    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e87d0aa1-0827-4945-900d-79f42a611524-kube-api-access-hrlng" (OuterVolumeSpecName: "kube-api-access-hrlng") pod "e87d0aa1-0827-4945-900d-79f42a611524" (UID: "e87d0aa1-0827-4945-900d-79f42a611524"). InnerVolumeSpecName "kube-api-access-hrlng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:43:23.017784 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:23.017752    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e87d0aa1-0827-4945-900d-79f42a611524-dshm\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:23.017784 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:23.017779    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrlng\" (UniqueName: \"kubernetes.io/projected/e87d0aa1-0827-4945-900d-79f42a611524-kube-api-access-hrlng\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:23.017784 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:23.017789    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e87d0aa1-0827-4945-900d-79f42a611524-tls-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:23.386831 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:23.386798    2551 generic.go:356] "Generic (PLEG): container finished" podID="e87d0aa1-0827-4945-900d-79f42a611524" containerID="3cb05b83821e3394a724c8850d909c694b7c1ba23ca795cecd8f175604f6f543" exitCode=0
Sep 10 13:43:23.386974 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:23.386841    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"
Sep 10 13:43:23.386974 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:23.386863    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx" event={"ID":"e87d0aa1-0827-4945-900d-79f42a611524","Type":"ContainerDied","Data":"3cb05b83821e3394a724c8850d909c694b7c1ba23ca795cecd8f175604f6f543"}
Sep 10 13:43:23.386974 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:23.386894    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx" event={"ID":"e87d0aa1-0827-4945-900d-79f42a611524","Type":"ContainerDied","Data":"691493bba287ed1c64d9f454692e644110c12a284dba2bdf1695907d4f08a22c"}
Sep 10 13:43:23.386974 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:23.386910    2551 scope.go:117] "RemoveContainer" containerID="3cb05b83821e3394a724c8850d909c694b7c1ba23ca795cecd8f175604f6f543"
Sep 10 13:43:23.395815 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:23.395799    2551 scope.go:117] "RemoveContainer" containerID="3cb05b83821e3394a724c8850d909c694b7c1ba23ca795cecd8f175604f6f543"
Sep 10 13:43:23.396096 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:43:23.396078    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cb05b83821e3394a724c8850d909c694b7c1ba23ca795cecd8f175604f6f543\": container with ID starting with 3cb05b83821e3394a724c8850d909c694b7c1ba23ca795cecd8f175604f6f543 not found: ID does not exist" containerID="3cb05b83821e3394a724c8850d909c694b7c1ba23ca795cecd8f175604f6f543"
Sep 10 13:43:23.396137 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:23.396103    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb05b83821e3394a724c8850d909c694b7c1ba23ca795cecd8f175604f6f543"} err="failed to get container status \"3cb05b83821e3394a724c8850d909c694b7c1ba23ca795cecd8f175604f6f543\": rpc error: code = NotFound desc = could not find container \"3cb05b83821e3394a724c8850d909c694b7c1ba23ca795cecd8f175604f6f543\": container with ID starting with 3cb05b83821e3394a724c8850d909c694b7c1ba23ca795cecd8f175604f6f543 not found: ID does not exist"
Sep 10 13:43:23.409562 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:23.409540    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"]
Sep 10 13:43:23.414065 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:23.414043    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-9469c838/e2e-lifecycle-model-9469c838-kserve-85bdf997bc-r4rxx"]
Sep 10 13:43:23.710312 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:23.710228    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e87d0aa1-0827-4945-900d-79f42a611524" path="/var/lib/kubelet/pods/e87d0aa1-0827-4945-900d-79f42a611524/volumes"
Sep 10 13:43:27.415557 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.415490    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs" podUID="cb50401b-3ee6-4097-a145-187a8cb40858" containerName="main" containerID="cri-o://ec610fe541e9ef50b5238f50ff93b55768cb2d8a0d316bafbe5b6d93c8d3838b" gracePeriod=45
Sep 10 13:43:27.666342 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.666275    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:43:27.767569 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.767536    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdwdz\" (UniqueName: \"kubernetes.io/projected/cb50401b-3ee6-4097-a145-187a8cb40858-kube-api-access-tdwdz\") pod \"cb50401b-3ee6-4097-a145-187a8cb40858\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") "
Sep 10 13:43:27.767766 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.767640    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-home\") pod \"cb50401b-3ee6-4097-a145-187a8cb40858\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") "
Sep 10 13:43:27.767766 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.767687    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb50401b-3ee6-4097-a145-187a8cb40858-tls-certs\") pod \"cb50401b-3ee6-4097-a145-187a8cb40858\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") "
Sep 10 13:43:27.767766 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.767718    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-model-cache\") pod \"cb50401b-3ee6-4097-a145-187a8cb40858\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") "
Sep 10 13:43:27.767766 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.767754    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-dshm\") pod \"cb50401b-3ee6-4097-a145-187a8cb40858\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") "
Sep 10 13:43:27.767991 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.767779    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-tmp-dir\") pod \"cb50401b-3ee6-4097-a145-187a8cb40858\" (UID: \"cb50401b-3ee6-4097-a145-187a8cb40858\") "
Sep 10 13:43:27.767991 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.767910    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-home" (OuterVolumeSpecName: "home") pod "cb50401b-3ee6-4097-a145-187a8cb40858" (UID: "cb50401b-3ee6-4097-a145-187a8cb40858"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:27.767991 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.767971    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-model-cache" (OuterVolumeSpecName: "model-cache") pod "cb50401b-3ee6-4097-a145-187a8cb40858" (UID: "cb50401b-3ee6-4097-a145-187a8cb40858"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:27.768197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.768168    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cb50401b-3ee6-4097-a145-187a8cb40858" (UID: "cb50401b-3ee6-4097-a145-187a8cb40858"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:27.768197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.768184    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-home\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:27.768349 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.768213    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-model-cache\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:27.769795 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.769774    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb50401b-3ee6-4097-a145-187a8cb40858-kube-api-access-tdwdz" (OuterVolumeSpecName: "kube-api-access-tdwdz") pod "cb50401b-3ee6-4097-a145-187a8cb40858" (UID: "cb50401b-3ee6-4097-a145-187a8cb40858"). InnerVolumeSpecName "kube-api-access-tdwdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:43:27.770249 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.770229    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb50401b-3ee6-4097-a145-187a8cb40858-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cb50401b-3ee6-4097-a145-187a8cb40858" (UID: "cb50401b-3ee6-4097-a145-187a8cb40858"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:43:27.770310 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.770241    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-dshm" (OuterVolumeSpecName: "dshm") pod "cb50401b-3ee6-4097-a145-187a8cb40858" (UID: "cb50401b-3ee6-4097-a145-187a8cb40858"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:27.868897 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.868869    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb50401b-3ee6-4097-a145-187a8cb40858-tls-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:27.868897 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.868894    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-dshm\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:27.869060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.868922    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb50401b-3ee6-4097-a145-187a8cb40858-tmp-dir\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:27.869060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:27.868931    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdwdz\" (UniqueName: \"kubernetes.io/projected/cb50401b-3ee6-4097-a145-187a8cb40858-kube-api-access-tdwdz\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:28.415713 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.415676    2551 generic.go:356] "Generic (PLEG): container finished" podID="cb50401b-3ee6-4097-a145-187a8cb40858" containerID="ec610fe541e9ef50b5238f50ff93b55768cb2d8a0d316bafbe5b6d93c8d3838b" exitCode=0
Sep 10 13:43:28.416175 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.415726    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"
Sep 10 13:43:28.416175 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.415759    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs" event={"ID":"cb50401b-3ee6-4097-a145-187a8cb40858","Type":"ContainerDied","Data":"ec610fe541e9ef50b5238f50ff93b55768cb2d8a0d316bafbe5b6d93c8d3838b"}
Sep 10 13:43:28.416175 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.415802    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs" event={"ID":"cb50401b-3ee6-4097-a145-187a8cb40858","Type":"ContainerDied","Data":"0ff85f9e803669a38e56a4c7eb442c446d6f8d78a79f66caed95294c4f569abc"}
Sep 10 13:43:28.416175 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.415819    2551 scope.go:117] "RemoveContainer" containerID="ec610fe541e9ef50b5238f50ff93b55768cb2d8a0d316bafbe5b6d93c8d3838b"
Sep 10 13:43:28.425117 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.425101    2551 scope.go:117] "RemoveContainer" containerID="ec610fe541e9ef50b5238f50ff93b55768cb2d8a0d316bafbe5b6d93c8d3838b"
Sep 10 13:43:28.425382 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:43:28.425367    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec610fe541e9ef50b5238f50ff93b55768cb2d8a0d316bafbe5b6d93c8d3838b\": container with ID starting with ec610fe541e9ef50b5238f50ff93b55768cb2d8a0d316bafbe5b6d93c8d3838b not found: ID does not exist" containerID="ec610fe541e9ef50b5238f50ff93b55768cb2d8a0d316bafbe5b6d93c8d3838b"
Sep 10 13:43:28.425444 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.425390    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec610fe541e9ef50b5238f50ff93b55768cb2d8a0d316bafbe5b6d93c8d3838b"} err="failed to get container status \"ec610fe541e9ef50b5238f50ff93b55768cb2d8a0d316bafbe5b6d93c8d3838b\": rpc error: code = NotFound desc = could not find container \"ec610fe541e9ef50b5238f50ff93b55768cb2d8a0d316bafbe5b6d93c8d3838b\": container with ID starting with ec610fe541e9ef50b5238f50ff93b55768cb2d8a0d316bafbe5b6d93c8d3838b not found: ID does not exist"
Sep 10 13:43:28.445755 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.445731    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"]
Sep 10 13:43:28.449283 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.449262    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/auth-test-model-767dce-kserve-84c74f475f-xn8gs"]
Sep 10 13:43:28.644481 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.644439    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k" podUID="c216272e-cb6f-481d-ab3f-7983dfc1939b" containerName="main" containerID="cri-o://3b6d382b1a644a9ef716f32883d292f10e11d67951cd67a445c818282bfbe25c" gracePeriod=45
Sep 10 13:43:28.898524 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.898502    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:43:28.978613 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.978558    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsmcs\" (UniqueName: \"kubernetes.io/projected/c216272e-cb6f-481d-ab3f-7983dfc1939b-kube-api-access-vsmcs\") pod \"c216272e-cb6f-481d-ab3f-7983dfc1939b\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") "
Sep 10 13:43:28.978613 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.978618    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-model-cache\") pod \"c216272e-cb6f-481d-ab3f-7983dfc1939b\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") "
Sep 10 13:43:28.978846 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.978640    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-home\") pod \"c216272e-cb6f-481d-ab3f-7983dfc1939b\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") "
Sep 10 13:43:28.978846 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.978679    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-tmp-dir\") pod \"c216272e-cb6f-481d-ab3f-7983dfc1939b\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") "
Sep 10 13:43:28.978846 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.978714    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-dshm\") pod \"c216272e-cb6f-481d-ab3f-7983dfc1939b\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") "
Sep 10 13:43:28.978846 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.978750    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c216272e-cb6f-481d-ab3f-7983dfc1939b-tls-certs\") pod \"c216272e-cb6f-481d-ab3f-7983dfc1939b\" (UID: \"c216272e-cb6f-481d-ab3f-7983dfc1939b\") "
Sep 10 13:43:28.979056 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.978973    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-model-cache" (OuterVolumeSpecName: "model-cache") pod "c216272e-cb6f-481d-ab3f-7983dfc1939b" (UID: "c216272e-cb6f-481d-ab3f-7983dfc1939b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:28.979056 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.978983    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-home" (OuterVolumeSpecName: "home") pod "c216272e-cb6f-481d-ab3f-7983dfc1939b" (UID: "c216272e-cb6f-481d-ab3f-7983dfc1939b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:28.979056 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.978996    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c216272e-cb6f-481d-ab3f-7983dfc1939b" (UID: "c216272e-cb6f-481d-ab3f-7983dfc1939b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:28.979166 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.979133    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-model-cache\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:28.979166 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.979156    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-home\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:28.979228 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.979169    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-tmp-dir\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:28.980938 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.980909    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-dshm" (OuterVolumeSpecName: "dshm") pod "c216272e-cb6f-481d-ab3f-7983dfc1939b" (UID: "c216272e-cb6f-481d-ab3f-7983dfc1939b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:28.981050 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.980960    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c216272e-cb6f-481d-ab3f-7983dfc1939b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c216272e-cb6f-481d-ab3f-7983dfc1939b" (UID: "c216272e-cb6f-481d-ab3f-7983dfc1939b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:43:28.981050 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:28.980975    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c216272e-cb6f-481d-ab3f-7983dfc1939b-kube-api-access-vsmcs" (OuterVolumeSpecName: "kube-api-access-vsmcs") pod "c216272e-cb6f-481d-ab3f-7983dfc1939b" (UID: "c216272e-cb6f-481d-ab3f-7983dfc1939b"). InnerVolumeSpecName "kube-api-access-vsmcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:43:29.080353 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:29.080289    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsmcs\" (UniqueName: \"kubernetes.io/projected/c216272e-cb6f-481d-ab3f-7983dfc1939b-kube-api-access-vsmcs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:29.080353 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:29.080313    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c216272e-cb6f-481d-ab3f-7983dfc1939b-dshm\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:29.080353 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:29.080323    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c216272e-cb6f-481d-ab3f-7983dfc1939b-tls-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:29.422638 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:29.422528    2551 generic.go:356] "Generic (PLEG): container finished" podID="c216272e-cb6f-481d-ab3f-7983dfc1939b" containerID="3b6d382b1a644a9ef716f32883d292f10e11d67951cd67a445c818282bfbe25c" exitCode=0
Sep 10 13:43:29.422638 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:29.422598    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"
Sep 10 13:43:29.422638 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:29.422614    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k" event={"ID":"c216272e-cb6f-481d-ab3f-7983dfc1939b","Type":"ContainerDied","Data":"3b6d382b1a644a9ef716f32883d292f10e11d67951cd67a445c818282bfbe25c"}
Sep 10 13:43:29.423248 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:29.422656    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k" event={"ID":"c216272e-cb6f-481d-ab3f-7983dfc1939b","Type":"ContainerDied","Data":"26ff1fe903c67ce27e53805e9ef0c8f577bded0c4d9cef1a616a1e7d4c3b8e96"}
Sep 10 13:43:29.423248 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:29.422674    2551 scope.go:117] "RemoveContainer" containerID="3b6d382b1a644a9ef716f32883d292f10e11d67951cd67a445c818282bfbe25c"
Sep 10 13:43:29.431853 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:29.431833    2551 scope.go:117] "RemoveContainer" containerID="3b6d382b1a644a9ef716f32883d292f10e11d67951cd67a445c818282bfbe25c"
Sep 10 13:43:29.432122 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:43:29.432104    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b6d382b1a644a9ef716f32883d292f10e11d67951cd67a445c818282bfbe25c\": container with ID starting with 3b6d382b1a644a9ef716f32883d292f10e11d67951cd67a445c818282bfbe25c not found: ID does not exist" containerID="3b6d382b1a644a9ef716f32883d292f10e11d67951cd67a445c818282bfbe25c"
Sep 10 13:43:29.432187 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:29.432134    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b6d382b1a644a9ef716f32883d292f10e11d67951cd67a445c818282bfbe25c"} err="failed to get container status \"3b6d382b1a644a9ef716f32883d292f10e11d67951cd67a445c818282bfbe25c\": rpc error: code = NotFound desc = could not find container \"3b6d382b1a644a9ef716f32883d292f10e11d67951cd67a445c818282bfbe25c\": container with ID starting with 3b6d382b1a644a9ef716f32883d292f10e11d67951cd67a445c818282bfbe25c not found: ID does not exist"
Sep 10 13:43:29.446129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:29.446102    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"]
Sep 10 13:43:29.454310 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:29.454280    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/auth-test-model-89be92-kserve-56746cdff5-gcv7k"]
Sep 10 13:43:29.709368 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:29.709293    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c216272e-cb6f-481d-ab3f-7983dfc1939b" path="/var/lib/kubelet/pods/c216272e-cb6f-481d-ab3f-7983dfc1939b/volumes"
Sep 10 13:43:29.709754 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:29.709739    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb50401b-3ee6-4097-a145-187a8cb40858" path="/var/lib/kubelet/pods/cb50401b-3ee6-4097-a145-187a8cb40858/volumes"
Sep 10 13:43:31.573413 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.573374    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"]
Sep 10 13:43:31.574813 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.574786    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb50401b-3ee6-4097-a145-187a8cb40858" containerName="main"
Sep 10 13:43:31.574990 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.574978    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb50401b-3ee6-4097-a145-187a8cb40858" containerName="main"
Sep 10 13:43:31.575076 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.575067    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c216272e-cb6f-481d-ab3f-7983dfc1939b" containerName="main"
Sep 10 13:43:31.575153 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.575145    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c216272e-cb6f-481d-ab3f-7983dfc1939b" containerName="main"
Sep 10 13:43:31.575224 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.575215    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e87d0aa1-0827-4945-900d-79f42a611524" containerName="main"
Sep 10 13:43:31.575295 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.575287    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e87d0aa1-0827-4945-900d-79f42a611524" containerName="main"
Sep 10 13:43:31.575386 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.575376    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3" containerName="test-curl-7787-bf82e8"
Sep 10 13:43:31.575454 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.575446    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3" containerName="test-curl-7787-bf82e8"
Sep 10 13:43:31.575748 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.575731    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c216272e-cb6f-481d-ab3f-7983dfc1939b" containerName="main"
Sep 10 13:43:31.575892 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.575879    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a70ed25-fdb7-4d17-9d0c-274d16c3bdc3" containerName="test-curl-7787-bf82e8"
Sep 10 13:43:31.575979 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.575969    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e87d0aa1-0827-4945-900d-79f42a611524" containerName="main"
Sep 10 13:43:31.576058 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.576049    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb50401b-3ee6-4097-a145-187a8cb40858" containerName="main"
Sep 10 13:43:31.582082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.582059    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.583067 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.583042    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"]
Sep 10 13:43:31.585053 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.585022    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-767dce\"/\"kube-root-ca.crt\""
Sep 10 13:43:31.585230 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.585188    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-767dce\"/\"default-dockercfg-zvw8j\""
Sep 10 13:43:31.585306 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.585241    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-767dce\"/\"openshift-service-ca.crt\""
Sep 10 13:43:31.585991 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.585967    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-767dce\"/\"sub-test-model-767dce-kserve-self-signed-certs\""
Sep 10 13:43:31.605567 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.605536    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-dshm\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.605686 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.605615    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-tls-certs\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.605731 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.605700    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-model-cache\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.605768 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.605749    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs6mh\" (UniqueName: \"kubernetes.io/projected/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-kube-api-access-vs6mh\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.605807 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.605772    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-tmp-dir\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.605807 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.605791    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-home\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.706229 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.706193    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-tls-certs\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.706464 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.706249    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-model-cache\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.706464 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.706283    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs6mh\" (UniqueName: \"kubernetes.io/projected/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-kube-api-access-vs6mh\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.706464 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.706308    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-tmp-dir\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.706464 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.706330    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-home\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.706464 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.706360    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-dshm\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.706775 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.706671    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-model-cache\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.706775 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.706714    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-tmp-dir\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.706885 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.706794    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-home\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.708709 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.708684    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-dshm\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.708931 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.708914    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-tls-certs\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.720074 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.719881    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs6mh\" (UniqueName: \"kubernetes.io/projected/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-kube-api-access-vs6mh\") pod \"sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") " pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:31.893178 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:31.893046    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:32.170301 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:32.170277    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"]
Sep 10 13:43:32.172129 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:43:32.172100    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7cd19f6d_0b1e_422b_ae75_5e8ee08ae8a9.slice/crio-a345d750c180ecb3df7345b29c88ab514f62e30695e93089bc22761371f6a069 WatchSource:0}: Error finding container a345d750c180ecb3df7345b29c88ab514f62e30695e93089bc22761371f6a069: Status 404 returned error can't find the container with id a345d750c180ecb3df7345b29c88ab514f62e30695e93089bc22761371f6a069
Sep 10 13:43:32.440245 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:32.440163    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l" event={"ID":"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9","Type":"ContainerStarted","Data":"db40d2ced04e645468455494266f3c8fec24c7e9b4b696c1a3dbc3a53549e127"}
Sep 10 13:43:32.440245 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:32.440203    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l" event={"ID":"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9","Type":"ContainerStarted","Data":"a345d750c180ecb3df7345b29c88ab514f62e30695e93089bc22761371f6a069"}
Sep 10 13:43:32.461670 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:32.461623    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l" podStartSLOduration=1.461606099 podStartE2EDuration="1.461606099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:43:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:43:32.458885684 +0000 UTC m=+1523.277924085" watchObservedRunningTime="2026-09-10 13:43:32.461606099 +0000 UTC m=+1523.280644492"
Sep 10 13:43:35.575868 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:35.575836    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:43:41.893215 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:41.893178    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:41.893215 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:41.893219    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:41.905283 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:41.905259    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:42.507660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:42.507629    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:43:51.459547 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:51.459509    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"]
Sep 10 13:43:51.460039 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:51.460008    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h" podUID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerName="istio-proxy" containerID="cri-o://dc9e31709917f92a19454920dd7898fa36595923b18b75d7d8eb5ace92c27652" gracePeriod=30
Sep 10 13:43:53.413749 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.413719    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"]
Sep 10 13:43:53.417190 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.417172    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.426169 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.426149    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-89be92\"/\"kube-root-ca.crt\""
Sep 10 13:43:53.426863 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.426844    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-89be92\"/\"openshift-service-ca.crt\""
Sep 10 13:43:53.426961 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.426884    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-89be92\"/\"sub-test-model-89be92-kserve-self-signed-certs\""
Sep 10 13:43:53.426961 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.426905    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-89be92\"/\"default-dockercfg-2ldvq\""
Sep 10 13:43:53.443475 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.443444    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"]
Sep 10 13:43:53.517306 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.517274    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-tls-certs\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.517431 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.517330    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-tmp-dir\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.517500 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.517456    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-model-cache\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.517500 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.517484    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltwx9\" (UniqueName: \"kubernetes.io/projected/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-kube-api-access-ltwx9\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.517621 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.517524    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-home\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.517621 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.517590    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-dshm\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.618208 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.618180    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-tls-certs\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.618391 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.618224    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-tmp-dir\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.618391 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.618282    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-model-cache\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.618391 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.618297    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltwx9\" (UniqueName: \"kubernetes.io/projected/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-kube-api-access-ltwx9\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.618391 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.618330    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-home\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.618642 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.618449    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-dshm\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.618703 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.618686    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-tmp-dir\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.618772 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.618755    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-home\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.618828 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.618798    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-model-cache\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.620600 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.620545    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-dshm\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.620821 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.620801    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-tls-certs\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.632598 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.632563    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltwx9\" (UniqueName: \"kubernetes.io/projected/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-kube-api-access-ltwx9\") pod \"sub-test-model-89be92-kserve-69fbd8986f-vt95n\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") " pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:53.727025 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:53.726994    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:43:54.007270 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:54.007116    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"]
Sep 10 13:43:54.009433 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:43:54.009405    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod541c6be7_eb44_41af_b0fa_a2f5e024c8f5.slice/crio-9475d863df3c77d65bc45b55f53071f0e19de740b826361b3220e908f2e2ea52 WatchSource:0}: Error finding container 9475d863df3c77d65bc45b55f53071f0e19de740b826361b3220e908f2e2ea52: Status 404 returned error can't find the container with id 9475d863df3c77d65bc45b55f53071f0e19de740b826361b3220e908f2e2ea52
Sep 10 13:43:54.561115 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:54.561076    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n" event={"ID":"541c6be7-eb44-41af-b0fa-a2f5e024c8f5","Type":"ContainerStarted","Data":"79ea61143bc2c22d70e81358bfbffcb9613e48cf321da6d9ca55012513837e76"}
Sep 10 13:43:54.561557 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:54.561123    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n" event={"ID":"541c6be7-eb44-41af-b0fa-a2f5e024c8f5","Type":"ContainerStarted","Data":"9475d863df3c77d65bc45b55f53071f0e19de740b826361b3220e908f2e2ea52"}
Sep 10 13:43:54.587181 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:54.587113    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n" podStartSLOduration=1.587095012 podStartE2EDuration="1.587095012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:43:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:43:54.586035455 +0000 UTC m=+1545.405073855" watchObservedRunningTime="2026-09-10 13:43:54.587095012 +0000 UTC m=+1545.406133422"
Sep 10 13:43:55.725922 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:55.725880    2551 patch_prober.go:28] interesting pod/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 13:43:55.726297 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:55.725948    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h" podUID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 13:43:56.712659 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.712634    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:43:56.852708 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.852680    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c28acdd-f70d-4f63-8376-50fe7398c58d-istiod-ca-cert\") pod \"3c28acdd-f70d-4f63-8376-50fe7398c58d\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") "
Sep 10 13:43:56.853102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.852725    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-credential-socket\") pod \"3c28acdd-f70d-4f63-8376-50fe7398c58d\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") "
Sep 10 13:43:56.853102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.852780    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmjd5\" (UniqueName: \"kubernetes.io/projected/3c28acdd-f70d-4f63-8376-50fe7398c58d-kube-api-access-hmjd5\") pod \"3c28acdd-f70d-4f63-8376-50fe7398c58d\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") "
Sep 10 13:43:56.853102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.852811    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-workload-socket\") pod \"3c28acdd-f70d-4f63-8376-50fe7398c58d\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") "
Sep 10 13:43:56.853102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.852834    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-envoy\") pod \"3c28acdd-f70d-4f63-8376-50fe7398c58d\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") "
Sep 10 13:43:56.853102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.852868    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-podinfo\") pod \"3c28acdd-f70d-4f63-8376-50fe7398c58d\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") "
Sep 10 13:43:56.853102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.852936    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-workload-certs\") pod \"3c28acdd-f70d-4f63-8376-50fe7398c58d\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") "
Sep 10 13:43:56.853102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.852970    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-token\") pod \"3c28acdd-f70d-4f63-8376-50fe7398c58d\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") "
Sep 10 13:43:56.853102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.852992    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-data\") pod \"3c28acdd-f70d-4f63-8376-50fe7398c58d\" (UID: \"3c28acdd-f70d-4f63-8376-50fe7398c58d\") "
Sep 10 13:43:56.853454 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.853097    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c28acdd-f70d-4f63-8376-50fe7398c58d" (UID: "3c28acdd-f70d-4f63-8376-50fe7398c58d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:56.853454 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.853162    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c28acdd-f70d-4f63-8376-50fe7398c58d" (UID: "3c28acdd-f70d-4f63-8376-50fe7398c58d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:56.853454 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.853198    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c28acdd-f70d-4f63-8376-50fe7398c58d" (UID: "3c28acdd-f70d-4f63-8376-50fe7398c58d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:56.853454 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.853298    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c28acdd-f70d-4f63-8376-50fe7398c58d" (UID: "3c28acdd-f70d-4f63-8376-50fe7398c58d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:56.853454 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.853397    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c28acdd-f70d-4f63-8376-50fe7398c58d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c28acdd-f70d-4f63-8376-50fe7398c58d" (UID: "3c28acdd-f70d-4f63-8376-50fe7398c58d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:43:56.853454 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.853419    2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-workload-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:56.853454 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.853438    2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-data\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:56.853454 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.853455    2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-credential-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:56.853789 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.853473    2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-workload-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:56.855174 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.855149    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c28acdd-f70d-4f63-8376-50fe7398c58d" (UID: "3c28acdd-f70d-4f63-8376-50fe7398c58d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:43:56.855258 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.855242    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c28acdd-f70d-4f63-8376-50fe7398c58d" (UID: "3c28acdd-f70d-4f63-8376-50fe7398c58d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 13:43:56.855526 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.855505    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c28acdd-f70d-4f63-8376-50fe7398c58d-kube-api-access-hmjd5" (OuterVolumeSpecName: "kube-api-access-hmjd5") pod "3c28acdd-f70d-4f63-8376-50fe7398c58d" (UID: "3c28acdd-f70d-4f63-8376-50fe7398c58d"). InnerVolumeSpecName "kube-api-access-hmjd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:43:56.855593 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.855550    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c28acdd-f70d-4f63-8376-50fe7398c58d" (UID: "3c28acdd-f70d-4f63-8376-50fe7398c58d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:43:56.954323 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.954290    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmjd5\" (UniqueName: \"kubernetes.io/projected/3c28acdd-f70d-4f63-8376-50fe7398c58d-kube-api-access-hmjd5\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:56.954323 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.954319    2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-envoy\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:56.954323 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.954328    2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-podinfo\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:56.954522 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.954336    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c28acdd-f70d-4f63-8376-50fe7398c58d-istio-token\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:56.954522 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:56.954345    2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c28acdd-f70d-4f63-8376-50fe7398c58d-istiod-ca-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:43:57.578256 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:57.578223    2551 generic.go:356] "Generic (PLEG): container finished" podID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerID="dc9e31709917f92a19454920dd7898fa36595923b18b75d7d8eb5ace92c27652" exitCode=0
Sep 10 13:43:57.578428 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:57.578267    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h" event={"ID":"3c28acdd-f70d-4f63-8376-50fe7398c58d","Type":"ContainerDied","Data":"dc9e31709917f92a19454920dd7898fa36595923b18b75d7d8eb5ace92c27652"}
Sep 10 13:43:57.578428 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:57.578278    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"
Sep 10 13:43:57.578428 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:57.578311    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h" event={"ID":"3c28acdd-f70d-4f63-8376-50fe7398c58d","Type":"ContainerDied","Data":"a674656a1333eabcfe1ae7b7b93e6a5752a6a31cf07a5b1190748874d2cbad20"}
Sep 10 13:43:57.578428 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:57.578333    2551 scope.go:117] "RemoveContainer" containerID="dc9e31709917f92a19454920dd7898fa36595923b18b75d7d8eb5ace92c27652"
Sep 10 13:43:57.587822 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:57.587804    2551 scope.go:117] "RemoveContainer" containerID="61f4477a757c86c7be3a9b912a76bfbe4af743cd71ceb0443cb4efcbd05711f4"
Sep 10 13:43:57.595721 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:57.595698    2551 scope.go:117] "RemoveContainer" containerID="dc9e31709917f92a19454920dd7898fa36595923b18b75d7d8eb5ace92c27652"
Sep 10 13:43:57.595978 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:43:57.595961    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc9e31709917f92a19454920dd7898fa36595923b18b75d7d8eb5ace92c27652\": container with ID starting with dc9e31709917f92a19454920dd7898fa36595923b18b75d7d8eb5ace92c27652 not found: ID does not exist" containerID="dc9e31709917f92a19454920dd7898fa36595923b18b75d7d8eb5ace92c27652"
Sep 10 13:43:57.596025 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:57.595985    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc9e31709917f92a19454920dd7898fa36595923b18b75d7d8eb5ace92c27652"} err="failed to get container status \"dc9e31709917f92a19454920dd7898fa36595923b18b75d7d8eb5ace92c27652\": rpc error: code = NotFound desc = could not find container \"dc9e31709917f92a19454920dd7898fa36595923b18b75d7d8eb5ace92c27652\": container with ID starting with dc9e31709917f92a19454920dd7898fa36595923b18b75d7d8eb5ace92c27652 not found: ID does not exist"
Sep 10 13:43:57.596025 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:57.596000    2551 scope.go:117] "RemoveContainer" containerID="61f4477a757c86c7be3a9b912a76bfbe4af743cd71ceb0443cb4efcbd05711f4"
Sep 10 13:43:57.596233 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:43:57.596217    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61f4477a757c86c7be3a9b912a76bfbe4af743cd71ceb0443cb4efcbd05711f4\": container with ID starting with 61f4477a757c86c7be3a9b912a76bfbe4af743cd71ceb0443cb4efcbd05711f4 not found: ID does not exist" containerID="61f4477a757c86c7be3a9b912a76bfbe4af743cd71ceb0443cb4efcbd05711f4"
Sep 10 13:43:57.596271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:57.596237    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61f4477a757c86c7be3a9b912a76bfbe4af743cd71ceb0443cb4efcbd05711f4"} err="failed to get container status \"61f4477a757c86c7be3a9b912a76bfbe4af743cd71ceb0443cb4efcbd05711f4\": rpc error: code = NotFound desc = could not find container \"61f4477a757c86c7be3a9b912a76bfbe4af743cd71ceb0443cb4efcbd05711f4\": container with ID starting with 61f4477a757c86c7be3a9b912a76bfbe4af743cd71ceb0443cb4efcbd05711f4 not found: ID does not exist"
Sep 10 13:43:57.602837 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:57.602817    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"]
Sep 10 13:43:57.609280 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:57.609258    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9469c838-openshift-default-6d456d9fdd-tbg5h"]
Sep 10 13:43:57.710821 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:57.710790    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c28acdd-f70d-4f63-8376-50fe7398c58d" path="/var/lib/kubelet/pods/3c28acdd-f70d-4f63-8376-50fe7398c58d/volumes"
Sep 10 13:43:58.185205 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:43:58.185168    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:44:03.728281 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:03.728245    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:44:03.728281 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:03.728291    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:44:03.740973 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:03.740946    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:44:04.626723 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:04.626688    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:44:09.855230 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:09.855200    2551 scope.go:117] "RemoveContainer" containerID="9fa4c75a41ac30622f59c54238d35da15d7cc7d3efc4abbc8c02b9d263410574"
Sep 10 13:44:09.862966 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:09.862948    2551 scope.go:117] "RemoveContainer" containerID="5d232a6b6e6d23da43ac2ca1b14cc4017b6655a476815033321ab514491d742b"
Sep 10 13:44:13.966759 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:13.966722    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:44:22.972011 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:22.971975    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:44:39.283371 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:39.283335    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:44:42.595962 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:42.595925    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7787-9e4ed7"]
Sep 10 13:44:42.596336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:42.596262    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerName="istio-proxy"
Sep 10 13:44:42.596336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:42.596274    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerName="istio-proxy"
Sep 10 13:44:42.596336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:42.596282    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerName="istio-proxy"
Sep 10 13:44:42.596336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:42.596288    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerName="istio-proxy"
Sep 10 13:44:42.596473 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:42.596361    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerName="istio-proxy"
Sep 10 13:44:42.599286 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:42.599268    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7787-9e4ed7"
Sep 10 13:44:42.601723 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:42.601703    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lpqwx\""
Sep 10 13:44:42.606340 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:42.606170    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7787-9e4ed7"]
Sep 10 13:44:42.763472 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:42.763444    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sblr\" (UniqueName: \"kubernetes.io/projected/25f0ee23-86f6-4707-ac28-ba2c4a15553a-kube-api-access-5sblr\") pod \"test-curl-7787-9e4ed7\" (UID: \"25f0ee23-86f6-4707-ac28-ba2c4a15553a\") " pod="opendatahub/test-curl-7787-9e4ed7"
Sep 10 13:44:42.864962 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:42.864891    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sblr\" (UniqueName: \"kubernetes.io/projected/25f0ee23-86f6-4707-ac28-ba2c4a15553a-kube-api-access-5sblr\") pod \"test-curl-7787-9e4ed7\" (UID: \"25f0ee23-86f6-4707-ac28-ba2c4a15553a\") " pod="opendatahub/test-curl-7787-9e4ed7"
Sep 10 13:44:42.873712 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:42.873690    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sblr\" (UniqueName: \"kubernetes.io/projected/25f0ee23-86f6-4707-ac28-ba2c4a15553a-kube-api-access-5sblr\") pod \"test-curl-7787-9e4ed7\" (UID: \"25f0ee23-86f6-4707-ac28-ba2c4a15553a\") " pod="opendatahub/test-curl-7787-9e4ed7"
Sep 10 13:44:42.909709 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:42.909688    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7787-9e4ed7"
Sep 10 13:44:43.177908 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:43.177886    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7787-9e4ed7"]
Sep 10 13:44:43.180035 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:44:43.180009    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25f0ee23_86f6_4707_ac28_ba2c4a15553a.slice/crio-162ba3ae4788962bf82ad7f692ec6bd7268016cc9c15e2e94d1a6c584c1141de WatchSource:0}: Error finding container 162ba3ae4788962bf82ad7f692ec6bd7268016cc9c15e2e94d1a6c584c1141de: Status 404 returned error can't find the container with id 162ba3ae4788962bf82ad7f692ec6bd7268016cc9c15e2e94d1a6c584c1141de
Sep 10 13:44:43.799958 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:43.799926    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7787-9e4ed7" event={"ID":"25f0ee23-86f6-4707-ac28-ba2c4a15553a","Type":"ContainerStarted","Data":"5b3174e2aafd20281f317129fb2814e25a43f76d2e5bc1d51d8b623c94c78b33"}
Sep 10 13:44:43.799958 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:43.799964    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7787-9e4ed7" event={"ID":"25f0ee23-86f6-4707-ac28-ba2c4a15553a","Type":"ContainerStarted","Data":"162ba3ae4788962bf82ad7f692ec6bd7268016cc9c15e2e94d1a6c584c1141de"}
Sep 10 13:44:43.821282 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:43.821238    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7787-9e4ed7" podStartSLOduration=1.821224606 podStartE2EDuration="1.821224606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:44:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:44:43.819347988 +0000 UTC m=+1594.638386392" watchObservedRunningTime="2026-09-10 13:44:43.821224606 +0000 UTC m=+1594.640263006"
Sep 10 13:44:44.426266 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:44.426232    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7787-9e4ed7"]
Sep 10 13:44:44.431427 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:44.431404    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7787-9e4ed7"]
Sep 10 13:44:44.541066 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:44.541036    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7787-38ae3b"]
Sep 10 13:44:44.541443 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:44.541429    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25f0ee23-86f6-4707-ac28-ba2c4a15553a" containerName="test-curl-7787-9e4ed7"
Sep 10 13:44:44.541492 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:44.541445    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="25f0ee23-86f6-4707-ac28-ba2c4a15553a" containerName="test-curl-7787-9e4ed7"
Sep 10 13:44:44.541530 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:44.541507    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c28acdd-f70d-4f63-8376-50fe7398c58d" containerName="istio-proxy"
Sep 10 13:44:44.541530 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:44.541518    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="25f0ee23-86f6-4707-ac28-ba2c4a15553a" containerName="test-curl-7787-9e4ed7"
Sep 10 13:44:44.544468 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:44.544452    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7787-38ae3b"
Sep 10 13:44:44.552403 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:44.552381    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7787-38ae3b"]
Sep 10 13:44:44.683264 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:44.683230    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j2n9\" (UniqueName: \"kubernetes.io/projected/3650d323-ce4b-4a92-bb82-eed0c59c020c-kube-api-access-9j2n9\") pod \"test-curl-7787-38ae3b\" (UID: \"3650d323-ce4b-4a92-bb82-eed0c59c020c\") " pod="opendatahub/test-curl-7787-38ae3b"
Sep 10 13:44:44.783880 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:44.783845    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j2n9\" (UniqueName: \"kubernetes.io/projected/3650d323-ce4b-4a92-bb82-eed0c59c020c-kube-api-access-9j2n9\") pod \"test-curl-7787-38ae3b\" (UID: \"3650d323-ce4b-4a92-bb82-eed0c59c020c\") " pod="opendatahub/test-curl-7787-38ae3b"
Sep 10 13:44:44.793550 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:44.793497    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j2n9\" (UniqueName: \"kubernetes.io/projected/3650d323-ce4b-4a92-bb82-eed0c59c020c-kube-api-access-9j2n9\") pod \"test-curl-7787-38ae3b\" (UID: \"3650d323-ce4b-4a92-bb82-eed0c59c020c\") " pod="opendatahub/test-curl-7787-38ae3b"
Sep 10 13:44:44.804753 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:44:44.804733    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7787-9e4ed7" container="test-curl-7787-9e4ed7"
Sep 10 13:44:44.854389 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:44.854369    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7787-38ae3b"
Sep 10 13:44:45.121625 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:45.121536    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7787-38ae3b"]
Sep 10 13:44:45.124868 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:44:45.124841    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3650d323_ce4b_4a92_bb82_eed0c59c020c.slice/crio-e7926e621d9c93f9ee36f5f061f20103c7640963aa85d09feabbebf3e0191ce5 WatchSource:0}: Error finding container e7926e621d9c93f9ee36f5f061f20103c7640963aa85d09feabbebf3e0191ce5: Status 404 returned error can't find the container with id e7926e621d9c93f9ee36f5f061f20103c7640963aa85d09feabbebf3e0191ce5
Sep 10 13:44:45.810952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:45.810919    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7787-38ae3b" event={"ID":"3650d323-ce4b-4a92-bb82-eed0c59c020c","Type":"ContainerStarted","Data":"f6eeb6bf8721020e76f92e3b14a1449d49a6bb54fb4ad286d11e9564d946f137"}
Sep 10 13:44:45.810952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:45.810959    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7787-38ae3b" event={"ID":"3650d323-ce4b-4a92-bb82-eed0c59c020c","Type":"ContainerStarted","Data":"e7926e621d9c93f9ee36f5f061f20103c7640963aa85d09feabbebf3e0191ce5"}
Sep 10 13:44:45.811420 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:45.811071    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7787-9e4ed7" podUID="25f0ee23-86f6-4707-ac28-ba2c4a15553a" containerName="test-curl-7787-9e4ed7" containerID="cri-o://5b3174e2aafd20281f317129fb2814e25a43f76d2e5bc1d51d8b623c94c78b33" gracePeriod=2
Sep 10 13:44:45.826565 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:45.826520    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7787-38ae3b" podStartSLOduration=1.8265045500000001 podStartE2EDuration="1.82650455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:44:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:44:45.825597835 +0000 UTC m=+1596.644636234" watchObservedRunningTime="2026-09-10 13:44:45.82650455 +0000 UTC m=+1596.645542950"
Sep 10 13:44:46.398252 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:46.398221    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7787-38ae3b"]
Sep 10 13:44:46.404047 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:46.404021    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7787-38ae3b"]
Sep 10 13:44:46.816124 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:44:46.816096    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7787-38ae3b" container="test-curl-7787-38ae3b"
Sep 10 13:44:47.820716 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:47.820685    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7787-38ae3b" podUID="3650d323-ce4b-4a92-bb82-eed0c59c020c" containerName="test-curl-7787-38ae3b" containerID="cri-o://f6eeb6bf8721020e76f92e3b14a1449d49a6bb54fb4ad286d11e9564d946f137" gracePeriod=2
Sep 10 13:44:47.959087 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:47.959062    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7787-9e4ed7"
Sep 10 13:44:48.118406 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.118339    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sblr\" (UniqueName: \"kubernetes.io/projected/25f0ee23-86f6-4707-ac28-ba2c4a15553a-kube-api-access-5sblr\") pod \"25f0ee23-86f6-4707-ac28-ba2c4a15553a\" (UID: \"25f0ee23-86f6-4707-ac28-ba2c4a15553a\") "
Sep 10 13:44:48.120255 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.120229    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25f0ee23-86f6-4707-ac28-ba2c4a15553a-kube-api-access-5sblr" (OuterVolumeSpecName: "kube-api-access-5sblr") pod "25f0ee23-86f6-4707-ac28-ba2c4a15553a" (UID: "25f0ee23-86f6-4707-ac28-ba2c4a15553a"). InnerVolumeSpecName "kube-api-access-5sblr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:44:48.219598 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.219554    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sblr\" (UniqueName: \"kubernetes.io/projected/25f0ee23-86f6-4707-ac28-ba2c4a15553a-kube-api-access-5sblr\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:44:48.429405 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.429325    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"]
Sep 10 13:44:48.547780 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.547752    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"]
Sep 10 13:44:48.548120 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.548108    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3650d323-ce4b-4a92-bb82-eed0c59c020c" containerName="test-curl-7787-38ae3b"
Sep 10 13:44:48.548164 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.548122    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3650d323-ce4b-4a92-bb82-eed0c59c020c" containerName="test-curl-7787-38ae3b"
Sep 10 13:44:48.548196 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.548189    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3650d323-ce4b-4a92-bb82-eed0c59c020c" containerName="test-curl-7787-38ae3b"
Sep 10 13:44:48.551024 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.551005    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.553876 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.553856    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a2d945-openshift-default-dockercfg-t5gt4\""
Sep 10 13:44:48.563903 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.563878    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"]
Sep 10 13:44:48.724949 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.724912    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-envoy\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.725121 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.724974    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-workload-socket\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.725121 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.725025    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-data\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.725202 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.725137    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-credential-socket\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.725202 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.725187    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67f60078-4b8c-4c34-b608-a8c065e8fee4-istiod-ca-cert\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.725272 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.725209    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjkhd\" (UniqueName: \"kubernetes.io/projected/67f60078-4b8c-4c34-b608-a8c065e8fee4-kube-api-access-hjkhd\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.725272 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.725227    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-podinfo\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.725272 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.725259    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-token\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.725386 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.725287    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-workload-certs\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.826358 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.826326    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-podinfo\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.826838 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.826367    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-token\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.826838 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.826403    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-workload-certs\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.826838 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.826477    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-envoy\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.826838 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.826567    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-workload-socket\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.826838 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.826613    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-data\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.826838 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.826677    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-credential-socket\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.826838 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.826750    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67f60078-4b8c-4c34-b608-a8c065e8fee4-istiod-ca-cert\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.826838 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.826798    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjkhd\" (UniqueName: \"kubernetes.io/projected/67f60078-4b8c-4c34-b608-a8c065e8fee4-kube-api-access-hjkhd\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.827276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.826841    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-workload-certs\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.827276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.827085    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-workload-socket\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.827395 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.827358    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-credential-socket\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.827521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.827494    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67f60078-4b8c-4c34-b608-a8c065e8fee4-istiod-ca-cert\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.827671 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.827645    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-data\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.827748 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.827661    2551 generic.go:356] "Generic (PLEG): container finished" podID="25f0ee23-86f6-4707-ac28-ba2c4a15553a" containerID="5b3174e2aafd20281f317129fb2814e25a43f76d2e5bc1d51d8b623c94c78b33" exitCode=137
Sep 10 13:44:48.827748 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.827728    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7787-9e4ed7"
Sep 10 13:44:48.827888 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.827756    2551 scope.go:117] "RemoveContainer" containerID="5b3174e2aafd20281f317129fb2814e25a43f76d2e5bc1d51d8b623c94c78b33"
Sep 10 13:44:48.829320 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.829301    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-podinfo\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.829677 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.829659    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-envoy\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.836551 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.836524    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjkhd\" (UniqueName: \"kubernetes.io/projected/67f60078-4b8c-4c34-b608-a8c065e8fee4-kube-api-access-hjkhd\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.836652 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.836524    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-token\") pod \"e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") " pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:48.852295 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:44:48.852198    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7787-9e4ed7\" is forbidden: User \"system:node:ip-10-0-135-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-117.ec2.internal' and this object" podUID="25f0ee23-86f6-4707-ac28-ba2c4a15553a" pod="opendatahub/test-curl-7787-9e4ed7"
Sep 10 13:44:48.852379 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.852331    2551 scope.go:117] "RemoveContainer" containerID="5b3174e2aafd20281f317129fb2814e25a43f76d2e5bc1d51d8b623c94c78b33"
Sep 10 13:44:48.852682 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:44:48.852662    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b3174e2aafd20281f317129fb2814e25a43f76d2e5bc1d51d8b623c94c78b33\": container with ID starting with 5b3174e2aafd20281f317129fb2814e25a43f76d2e5bc1d51d8b623c94c78b33 not found: ID does not exist" containerID="5b3174e2aafd20281f317129fb2814e25a43f76d2e5bc1d51d8b623c94c78b33"
Sep 10 13:44:48.852749 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.852690    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3174e2aafd20281f317129fb2814e25a43f76d2e5bc1d51d8b623c94c78b33"} err="failed to get container status \"5b3174e2aafd20281f317129fb2814e25a43f76d2e5bc1d51d8b623c94c78b33\": rpc error: code = NotFound desc = could not find container \"5b3174e2aafd20281f317129fb2814e25a43f76d2e5bc1d51d8b623c94c78b33\": container with ID starting with 5b3174e2aafd20281f317129fb2814e25a43f76d2e5bc1d51d8b623c94c78b33 not found: ID does not exist"
Sep 10 13:44:48.860924 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:48.860905    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:49.139651 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:49.139615    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"]
Sep 10 13:44:49.141806 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:44:49.141785    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67f60078_4b8c_4c34_b608_a8c065e8fee4.slice/crio-3c2daf227db3c5b1e8d722e04ae0a79529dce967ab1e7102e9203f81766a7ba6 WatchSource:0}: Error finding container 3c2daf227db3c5b1e8d722e04ae0a79529dce967ab1e7102e9203f81766a7ba6: Status 404 returned error can't find the container with id 3c2daf227db3c5b1e8d722e04ae0a79529dce967ab1e7102e9203f81766a7ba6
Sep 10 13:44:49.144590 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:49.144539    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:44:49.144691 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:49.144634    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:44:49.144691 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:49.144680    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:44:49.357107 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:49.357077    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"]
Sep 10 13:44:49.377883 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:44:49.377848    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7787-9e4ed7\" is forbidden: User \"system:node:ip-10-0-135-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-117.ec2.internal' and this object" podUID="25f0ee23-86f6-4707-ac28-ba2c4a15553a" pod="opendatahub/test-curl-7787-9e4ed7"
Sep 10 13:44:49.708298 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:44:49.708261    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7787-9e4ed7\" is forbidden: User \"system:node:ip-10-0-135-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-117.ec2.internal' and this object" podUID="25f0ee23-86f6-4707-ac28-ba2c4a15553a" pod="opendatahub/test-curl-7787-9e4ed7"
Sep 10 13:44:49.709821 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:49.709799    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25f0ee23-86f6-4707-ac28-ba2c4a15553a" path="/var/lib/kubelet/pods/25f0ee23-86f6-4707-ac28-ba2c4a15553a/volumes"
Sep 10 13:44:49.836003 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:49.835972    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r" event={"ID":"67f60078-4b8c-4c34-b608-a8c065e8fee4","Type":"ContainerStarted","Data":"2943f6752d03ae8988d7d09395d665726dd563cfec45ca2711ea3ecf66d2a1be"}
Sep 10 13:44:49.836384 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:49.836021    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r" event={"ID":"67f60078-4b8c-4c34-b608-a8c065e8fee4","Type":"ContainerStarted","Data":"3c2daf227db3c5b1e8d722e04ae0a79529dce967ab1e7102e9203f81766a7ba6"}
Sep 10 13:44:49.860711 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:49.860670    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r" podStartSLOduration=1.860657102 podStartE2EDuration="1.860657102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:44:49.858535828 +0000 UTC m=+1600.677574229" watchObservedRunningTime="2026-09-10 13:44:49.860657102 +0000 UTC m=+1600.679695502"
Sep 10 13:44:49.861204 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:49.861187    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:49.967766 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:49.967709    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7787-38ae3b"
Sep 10 13:44:50.137962 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:50.137931    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9j2n9\" (UniqueName: \"kubernetes.io/projected/3650d323-ce4b-4a92-bb82-eed0c59c020c-kube-api-access-9j2n9\") pod \"3650d323-ce4b-4a92-bb82-eed0c59c020c\" (UID: \"3650d323-ce4b-4a92-bb82-eed0c59c020c\") "
Sep 10 13:44:50.139959 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:50.139933    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3650d323-ce4b-4a92-bb82-eed0c59c020c-kube-api-access-9j2n9" (OuterVolumeSpecName: "kube-api-access-9j2n9") pod "3650d323-ce4b-4a92-bb82-eed0c59c020c" (UID: "3650d323-ce4b-4a92-bb82-eed0c59c020c"). InnerVolumeSpecName "kube-api-access-9j2n9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:44:50.239266 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:50.239235    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j2n9\" (UniqueName: \"kubernetes.io/projected/3650d323-ce4b-4a92-bb82-eed0c59c020c-kube-api-access-9j2n9\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:44:50.843775 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:50.843735    2551 generic.go:356] "Generic (PLEG): container finished" podID="3650d323-ce4b-4a92-bb82-eed0c59c020c" containerID="f6eeb6bf8721020e76f92e3b14a1449d49a6bb54fb4ad286d11e9564d946f137" exitCode=137
Sep 10 13:44:50.844212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:50.843780    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7787-38ae3b"
Sep 10 13:44:50.844212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:50.843823    2551 scope.go:117] "RemoveContainer" containerID="f6eeb6bf8721020e76f92e3b14a1449d49a6bb54fb4ad286d11e9564d946f137"
Sep 10 13:44:50.852731 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:50.852714    2551 scope.go:117] "RemoveContainer" containerID="f6eeb6bf8721020e76f92e3b14a1449d49a6bb54fb4ad286d11e9564d946f137"
Sep 10 13:44:50.852977 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:44:50.852959    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6eeb6bf8721020e76f92e3b14a1449d49a6bb54fb4ad286d11e9564d946f137\": container with ID starting with f6eeb6bf8721020e76f92e3b14a1449d49a6bb54fb4ad286d11e9564d946f137 not found: ID does not exist" containerID="f6eeb6bf8721020e76f92e3b14a1449d49a6bb54fb4ad286d11e9564d946f137"
Sep 10 13:44:50.853034 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:50.852987    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6eeb6bf8721020e76f92e3b14a1449d49a6bb54fb4ad286d11e9564d946f137"} err="failed to get container status \"f6eeb6bf8721020e76f92e3b14a1449d49a6bb54fb4ad286d11e9564d946f137\": rpc error: code = NotFound desc = could not find container \"f6eeb6bf8721020e76f92e3b14a1449d49a6bb54fb4ad286d11e9564d946f137\": container with ID starting with f6eeb6bf8721020e76f92e3b14a1449d49a6bb54fb4ad286d11e9564d946f137 not found: ID does not exist"
Sep 10 13:44:50.854561 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:44:50.854540    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7787-38ae3b\" is forbidden: User \"system:node:ip-10-0-135-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-117.ec2.internal' and this object" podUID="3650d323-ce4b-4a92-bb82-eed0c59c020c" pod="opendatahub/test-curl-7787-38ae3b"
Sep 10 13:44:50.867598 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:50.867566    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:50.870187 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:44:50.870162    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7787-38ae3b\" is forbidden: User \"system:node:ip-10-0-135-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-117.ec2.internal' and this object" podUID="3650d323-ce4b-4a92-bb82-eed0c59c020c" pod="opendatahub/test-curl-7787-38ae3b"
Sep 10 13:44:51.709983 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:51.709942    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3650d323-ce4b-4a92-bb82-eed0c59c020c" path="/var/lib/kubelet/pods/3650d323-ce4b-4a92-bb82-eed0c59c020c/volumes"
Sep 10 13:44:51.856035 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:51.856008    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:51.856969 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:51.856946    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:44:52.668714 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:44:52.668681    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:45:00.137851 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.137817    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817465-kp5xh"]
Sep 10 13:45:00.142655 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.142630    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-kp5xh"
Sep 10 13:45:00.145304 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.145271    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-89be92-29817465-bbpfh"]
Sep 10 13:45:00.146249 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.146226    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:45:00.148825 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.148803    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-767dce-29817465-hjsh2"]
Sep 10 13:45:00.148947 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.148895    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-89be92-29817465-bbpfh"
Sep 10 13:45:00.152271 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.152250    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-767dce-29817465-hjsh2"
Sep 10 13:45:00.154801 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.154783    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-kp5xh"
Sep 10 13:45:00.160913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.160326    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817465-kp5xh"]
Sep 10 13:45:00.165089 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.165033    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-89be92-29817465-bbpfh"]
Sep 10 13:45:00.166518 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.165794    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-89be92-29817465-bbpfh"
Sep 10 13:45:00.168974 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.168951    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-767dce-29817465-hjsh2"]
Sep 10 13:45:00.208239 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.208060    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-767dce-29817465-hjsh2"
Sep 10 13:45:00.453567 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.453538    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817465-kp5xh"]
Sep 10 13:45:00.454190 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:45:00.454162    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7c11da9_02cd_4291_8c97_c4e012a7acb7.slice/crio-5bee98c1e1288521605fbec8e6d9edfbaf6a3fea6528683cb989e6eb6439ffb7 WatchSource:0}: Error finding container 5bee98c1e1288521605fbec8e6d9edfbaf6a3fea6528683cb989e6eb6439ffb7: Status 404 returned error can't find the container with id 5bee98c1e1288521605fbec8e6d9edfbaf6a3fea6528683cb989e6eb6439ffb7
Sep 10 13:45:00.475871 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.475850    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-89be92-29817465-bbpfh"]
Sep 10 13:45:00.485558 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:45:00.485532    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod984544c4_7d1d_444f_8643_9835ca6a4aec.slice/crio-2da2dca64c42019f75f6b589bed4a9e009a1405126ae3ef841ad3fed481b29bc WatchSource:0}: Error finding container 2da2dca64c42019f75f6b589bed4a9e009a1405126ae3ef841ad3fed481b29bc: Status 404 returned error can't find the container with id 2da2dca64c42019f75f6b589bed4a9e009a1405126ae3ef841ad3fed481b29bc
Sep 10 13:45:00.525423 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:45:00.525396    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf8bbef1_caf2_4a08_800c_bcad76e29ea2.slice/crio-efa16219680fb30f759ed0fc4ff3f27dbfc88285b0f20841f0618ab96a2f690c WatchSource:0}: Error finding container efa16219680fb30f759ed0fc4ff3f27dbfc88285b0f20841f0618ab96a2f690c: Status 404 returned error can't find the container with id efa16219680fb30f759ed0fc4ff3f27dbfc88285b0f20841f0618ab96a2f690c
Sep 10 13:45:00.525695 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.525677    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-767dce-29817465-hjsh2"]
Sep 10 13:45:00.903934 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.903894    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-767dce-29817465-hjsh2" event={"ID":"bf8bbef1-caf2-4a08-800c-bcad76e29ea2","Type":"ContainerStarted","Data":"34a2da22da6080844b4ddbe755adf69d33bff57bfbacaf352b16c772be6b77c8"}
Sep 10 13:45:00.904110 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.903962    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-767dce-29817465-hjsh2" event={"ID":"bf8bbef1-caf2-4a08-800c-bcad76e29ea2","Type":"ContainerStarted","Data":"efa16219680fb30f759ed0fc4ff3f27dbfc88285b0f20841f0618ab96a2f690c"}
Sep 10 13:45:00.905661 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.905629    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-89be92-29817465-bbpfh" event={"ID":"984544c4-7d1d-444f-8643-9835ca6a4aec","Type":"ContainerStarted","Data":"d27cefb371abcce15bc17ccb1714893b25c3c0d4214fa03b1fac1940d461474a"}
Sep 10 13:45:00.905815 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.905684    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-89be92-29817465-bbpfh" event={"ID":"984544c4-7d1d-444f-8643-9835ca6a4aec","Type":"ContainerStarted","Data":"2da2dca64c42019f75f6b589bed4a9e009a1405126ae3ef841ad3fed481b29bc"}
Sep 10 13:45:00.907162 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.907138    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-kp5xh" event={"ID":"a7c11da9-02cd-4291-8c97-c4e012a7acb7","Type":"ContainerStarted","Data":"550c15277c1d9bd9d770ab0de478d77f8f74015cfdc805159444eb123de66fad"}
Sep 10 13:45:00.907309 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.907178    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-kp5xh" event={"ID":"a7c11da9-02cd-4291-8c97-c4e012a7acb7","Type":"ContainerStarted","Data":"5bee98c1e1288521605fbec8e6d9edfbaf6a3fea6528683cb989e6eb6439ffb7"}
Sep 10 13:45:00.927668 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.927617    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-767dce-29817465-hjsh2" podStartSLOduration=0.927604704 podStartE2EDuration="927.604704ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:00.92500854 +0000 UTC m=+1611.744046939" watchObservedRunningTime="2026-09-10 13:45:00.927604704 +0000 UTC m=+1611.746643138"
Sep 10 13:45:00.943821 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.943777    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-kp5xh" podStartSLOduration=0.943764124 podStartE2EDuration="943.764124ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:00.941930959 +0000 UTC m=+1611.760969360" watchObservedRunningTime="2026-09-10 13:45:00.943764124 +0000 UTC m=+1611.762802524"
Sep 10 13:45:00.959720 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:00.959668    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-89be92-29817465-bbpfh" podStartSLOduration=0.959651872 podStartE2EDuration="959.651872ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:00.957258237 +0000 UTC m=+1611.776296636" watchObservedRunningTime="2026-09-10 13:45:00.959651872 +0000 UTC m=+1611.778690273"
Sep 10 13:45:01.914261 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:01.914131    2551 generic.go:356] "Generic (PLEG): container finished" podID="bf8bbef1-caf2-4a08-800c-bcad76e29ea2" containerID="34a2da22da6080844b4ddbe755adf69d33bff57bfbacaf352b16c772be6b77c8" exitCode=0
Sep 10 13:45:01.914261 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:01.914226    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-767dce-29817465-hjsh2" event={"ID":"bf8bbef1-caf2-4a08-800c-bcad76e29ea2","Type":"ContainerDied","Data":"34a2da22da6080844b4ddbe755adf69d33bff57bfbacaf352b16c772be6b77c8"}
Sep 10 13:45:01.915970 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:01.915944    2551 generic.go:356] "Generic (PLEG): container finished" podID="984544c4-7d1d-444f-8643-9835ca6a4aec" containerID="d27cefb371abcce15bc17ccb1714893b25c3c0d4214fa03b1fac1940d461474a" exitCode=0
Sep 10 13:45:01.916103 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:01.915990    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-89be92-29817465-bbpfh" event={"ID":"984544c4-7d1d-444f-8643-9835ca6a4aec","Type":"ContainerDied","Data":"d27cefb371abcce15bc17ccb1714893b25c3c0d4214fa03b1fac1940d461474a"}
Sep 10 13:45:01.917630 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:01.917607    2551 generic.go:356] "Generic (PLEG): container finished" podID="a7c11da9-02cd-4291-8c97-c4e012a7acb7" containerID="550c15277c1d9bd9d770ab0de478d77f8f74015cfdc805159444eb123de66fad" exitCode=0
Sep 10 13:45:01.917713 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:01.917630    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-kp5xh" event={"ID":"a7c11da9-02cd-4291-8c97-c4e012a7acb7","Type":"ContainerDied","Data":"550c15277c1d9bd9d770ab0de478d77f8f74015cfdc805159444eb123de66fad"}
Sep 10 13:45:03.107912 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.107885    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-767dce-29817465-hjsh2"
Sep 10 13:45:03.171644 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.166037    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-kp5xh"
Sep 10 13:45:03.174771 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.174670    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-89be92-29817465-bbpfh"
Sep 10 13:45:03.453358 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.453255    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l" podUID="7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9" containerName="main" containerID="cri-o://db40d2ced04e645468455494266f3c8fec24c7e9b4b696c1a3dbc3a53549e127" gracePeriod=45
Sep 10 13:45:03.709934 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.709416    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:45:03.782401 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.782369    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vs6mh\" (UniqueName: \"kubernetes.io/projected/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-kube-api-access-vs6mh\") pod \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") "
Sep 10 13:45:03.782565 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.782439    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-home\") pod \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") "
Sep 10 13:45:03.782565 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.782507    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-dshm\") pod \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") "
Sep 10 13:45:03.782565 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.782533    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-tls-certs\") pod \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") "
Sep 10 13:45:03.782565 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.782555    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-tmp-dir\") pod \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") "
Sep 10 13:45:03.782804 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.782593    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-model-cache\") pod \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\" (UID: \"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9\") "
Sep 10 13:45:03.782804 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.782750    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-home" (OuterVolumeSpecName: "home") pod "7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9" (UID: "7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:45:03.782804 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.782764    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9" (UID: "7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:45:03.783004 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.782967    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-model-cache" (OuterVolumeSpecName: "model-cache") pod "7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9" (UID: "7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:45:03.783088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.783012    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-home\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:45:03.783088 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.783028    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-tmp-dir\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:45:03.784683 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.784653    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9" (UID: "7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:45:03.784780 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.784689    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-dshm" (OuterVolumeSpecName: "dshm") pod "7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9" (UID: "7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:45:03.785040 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.785021    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-kube-api-access-vs6mh" (OuterVolumeSpecName: "kube-api-access-vs6mh") pod "7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9" (UID: "7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9"). InnerVolumeSpecName "kube-api-access-vs6mh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:45:03.884347 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.884313    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vs6mh\" (UniqueName: \"kubernetes.io/projected/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-kube-api-access-vs6mh\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:45:03.884347 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.884341    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-dshm\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:45:03.884514 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.884354    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-tls-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:45:03.884514 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.884367    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9-model-cache\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:45:03.928639 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.928609    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-767dce-29817465-hjsh2" event={"ID":"bf8bbef1-caf2-4a08-800c-bcad76e29ea2","Type":"ContainerDied","Data":"efa16219680fb30f759ed0fc4ff3f27dbfc88285b0f20841f0618ab96a2f690c"}
Sep 10 13:45:03.928761 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.928661    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="efa16219680fb30f759ed0fc4ff3f27dbfc88285b0f20841f0618ab96a2f690c"
Sep 10 13:45:03.928761 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.928623    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-767dce-29817465-hjsh2"
Sep 10 13:45:03.930060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.930043    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-89be92-29817465-bbpfh"
Sep 10 13:45:03.930165 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.930081    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-89be92-29817465-bbpfh" event={"ID":"984544c4-7d1d-444f-8643-9835ca6a4aec","Type":"ContainerDied","Data":"2da2dca64c42019f75f6b589bed4a9e009a1405126ae3ef841ad3fed481b29bc"}
Sep 10 13:45:03.930165 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.930116    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2da2dca64c42019f75f6b589bed4a9e009a1405126ae3ef841ad3fed481b29bc"
Sep 10 13:45:03.931532 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.931516    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-kp5xh"
Sep 10 13:45:03.931649 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.931537    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817465-kp5xh" event={"ID":"a7c11da9-02cd-4291-8c97-c4e012a7acb7","Type":"ContainerDied","Data":"5bee98c1e1288521605fbec8e6d9edfbaf6a3fea6528683cb989e6eb6439ffb7"}
Sep 10 13:45:03.931649 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.931599    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5bee98c1e1288521605fbec8e6d9edfbaf6a3fea6528683cb989e6eb6439ffb7"
Sep 10 13:45:03.933056 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.933033    2551 generic.go:356] "Generic (PLEG): container finished" podID="7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9" containerID="db40d2ced04e645468455494266f3c8fec24c7e9b4b696c1a3dbc3a53549e127" exitCode=0
Sep 10 13:45:03.933144 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.933070    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l" event={"ID":"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9","Type":"ContainerDied","Data":"db40d2ced04e645468455494266f3c8fec24c7e9b4b696c1a3dbc3a53549e127"}
Sep 10 13:45:03.933144 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.933101    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"
Sep 10 13:45:03.933144 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.933108    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l" event={"ID":"7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9","Type":"ContainerDied","Data":"a345d750c180ecb3df7345b29c88ab514f62e30695e93089bc22761371f6a069"}
Sep 10 13:45:03.933144 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.933135    2551 scope.go:117] "RemoveContainer" containerID="db40d2ced04e645468455494266f3c8fec24c7e9b4b696c1a3dbc3a53549e127"
Sep 10 13:45:03.941867 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.941854    2551 scope.go:117] "RemoveContainer" containerID="db40d2ced04e645468455494266f3c8fec24c7e9b4b696c1a3dbc3a53549e127"
Sep 10 13:45:03.942117 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:45:03.942102    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db40d2ced04e645468455494266f3c8fec24c7e9b4b696c1a3dbc3a53549e127\": container with ID starting with db40d2ced04e645468455494266f3c8fec24c7e9b4b696c1a3dbc3a53549e127 not found: ID does not exist" containerID="db40d2ced04e645468455494266f3c8fec24c7e9b4b696c1a3dbc3a53549e127"
Sep 10 13:45:03.942158 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.942122    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db40d2ced04e645468455494266f3c8fec24c7e9b4b696c1a3dbc3a53549e127"} err="failed to get container status \"db40d2ced04e645468455494266f3c8fec24c7e9b4b696c1a3dbc3a53549e127\": rpc error: code = NotFound desc = could not find container \"db40d2ced04e645468455494266f3c8fec24c7e9b4b696c1a3dbc3a53549e127\": container with ID starting with db40d2ced04e645468455494266f3c8fec24c7e9b4b696c1a3dbc3a53549e127 not found: ID does not exist"
Sep 10 13:45:03.969680 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.969548    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"]
Sep 10 13:45:03.971995 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:03.971969    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/sub-test-model-767dce-kserve-5f46ccd6c5-d7t5l"]
Sep 10 13:45:04.376292 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.376245    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n" podUID="541c6be7-eb44-41af-b0fa-a2f5e024c8f5" containerName="main" containerID="cri-o://79ea61143bc2c22d70e81358bfbffcb9613e48cf321da6d9ca55012513837e76" gracePeriod=45
Sep 10 13:45:04.637753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.637698    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:45:04.690834 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.690794    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-home\") pod \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") "
Sep 10 13:45:04.691024 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.690861    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-model-cache\") pod \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") "
Sep 10 13:45:04.691024 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.690940    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-dshm\") pod \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") "
Sep 10 13:45:04.691153 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.691023    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltwx9\" (UniqueName: \"kubernetes.io/projected/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-kube-api-access-ltwx9\") pod \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") "
Sep 10 13:45:04.691153 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.691049    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-tmp-dir\") pod \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") "
Sep 10 13:45:04.691153 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.691051    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-model-cache" (OuterVolumeSpecName: "model-cache") pod "541c6be7-eb44-41af-b0fa-a2f5e024c8f5" (UID: "541c6be7-eb44-41af-b0fa-a2f5e024c8f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:45:04.691153 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.691070    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-home" (OuterVolumeSpecName: "home") pod "541c6be7-eb44-41af-b0fa-a2f5e024c8f5" (UID: "541c6be7-eb44-41af-b0fa-a2f5e024c8f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:45:04.691153 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.691084    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-tls-certs\") pod \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\" (UID: \"541c6be7-eb44-41af-b0fa-a2f5e024c8f5\") "
Sep 10 13:45:04.691417 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.691236    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "541c6be7-eb44-41af-b0fa-a2f5e024c8f5" (UID: "541c6be7-eb44-41af-b0fa-a2f5e024c8f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:45:04.691480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.691432    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-home\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:45:04.691480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.691448    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-model-cache\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:45:04.691480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.691462    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-tmp-dir\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:45:04.693150 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.693121    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-kube-api-access-ltwx9" (OuterVolumeSpecName: "kube-api-access-ltwx9") pod "541c6be7-eb44-41af-b0fa-a2f5e024c8f5" (UID: "541c6be7-eb44-41af-b0fa-a2f5e024c8f5"). InnerVolumeSpecName "kube-api-access-ltwx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:45:04.693254 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.693149    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "541c6be7-eb44-41af-b0fa-a2f5e024c8f5" (UID: "541c6be7-eb44-41af-b0fa-a2f5e024c8f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:45:04.693254 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.693159    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-dshm" (OuterVolumeSpecName: "dshm") pod "541c6be7-eb44-41af-b0fa-a2f5e024c8f5" (UID: "541c6be7-eb44-41af-b0fa-a2f5e024c8f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:45:04.792538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.792509    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-dshm\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:45:04.792538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.792534    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltwx9\" (UniqueName: \"kubernetes.io/projected/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-kube-api-access-ltwx9\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:45:04.792538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.792543    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541c6be7-eb44-41af-b0fa-a2f5e024c8f5-tls-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:45:04.940402 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.940315    2551 generic.go:356] "Generic (PLEG): container finished" podID="541c6be7-eb44-41af-b0fa-a2f5e024c8f5" containerID="79ea61143bc2c22d70e81358bfbffcb9613e48cf321da6d9ca55012513837e76" exitCode=0
Sep 10 13:45:04.940402 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.940369    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"
Sep 10 13:45:04.940612 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.940399    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n" event={"ID":"541c6be7-eb44-41af-b0fa-a2f5e024c8f5","Type":"ContainerDied","Data":"79ea61143bc2c22d70e81358bfbffcb9613e48cf321da6d9ca55012513837e76"}
Sep 10 13:45:04.940612 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.940439    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n" event={"ID":"541c6be7-eb44-41af-b0fa-a2f5e024c8f5","Type":"ContainerDied","Data":"9475d863df3c77d65bc45b55f53071f0e19de740b826361b3220e908f2e2ea52"}
Sep 10 13:45:04.940612 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.940456    2551 scope.go:117] "RemoveContainer" containerID="79ea61143bc2c22d70e81358bfbffcb9613e48cf321da6d9ca55012513837e76"
Sep 10 13:45:04.952276 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.952250    2551 scope.go:117] "RemoveContainer" containerID="79ea61143bc2c22d70e81358bfbffcb9613e48cf321da6d9ca55012513837e76"
Sep 10 13:45:04.952708 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:45:04.952680    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79ea61143bc2c22d70e81358bfbffcb9613e48cf321da6d9ca55012513837e76\": container with ID starting with 79ea61143bc2c22d70e81358bfbffcb9613e48cf321da6d9ca55012513837e76 not found: ID does not exist" containerID="79ea61143bc2c22d70e81358bfbffcb9613e48cf321da6d9ca55012513837e76"
Sep 10 13:45:04.952812 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.952716    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79ea61143bc2c22d70e81358bfbffcb9613e48cf321da6d9ca55012513837e76"} err="failed to get container status \"79ea61143bc2c22d70e81358bfbffcb9613e48cf321da6d9ca55012513837e76\": rpc error: code = NotFound desc = could not find container \"79ea61143bc2c22d70e81358bfbffcb9613e48cf321da6d9ca55012513837e76\": container with ID starting with 79ea61143bc2c22d70e81358bfbffcb9613e48cf321da6d9ca55012513837e76 not found: ID does not exist"
Sep 10 13:45:04.970646 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.970617    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"]
Sep 10 13:45:04.974722 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:04.974700    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n"]
Sep 10 13:45:05.716308 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:05.716266    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="541c6be7-eb44-41af-b0fa-a2f5e024c8f5" path="/var/lib/kubelet/pods/541c6be7-eb44-41af-b0fa-a2f5e024c8f5/volumes"
Sep 10 13:45:05.716854 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:05.716834    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9" path="/var/lib/kubelet/pods/7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9/volumes"
Sep 10 13:45:05.966893 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:05.966796    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:45:09.616510 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:09.616473    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w3-89be92/sub-test-model-89be92-kserve-69fbd8986f-vt95n" podUID="541c6be7-eb44-41af-b0fa-a2f5e024c8f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 10 13:45:11.982058 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:11.982028    2551 generic.go:356] "Generic (PLEG): container finished" podID="67f60078-4b8c-4c34-b608-a8c065e8fee4" containerID="2943f6752d03ae8988d7d09395d665726dd563cfec45ca2711ea3ecf66d2a1be" exitCode=0
Sep 10 13:45:11.982440 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:11.982097    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r" event={"ID":"67f60078-4b8c-4c34-b608-a8c065e8fee4","Type":"ContainerDied","Data":"2943f6752d03ae8988d7d09395d665726dd563cfec45ca2711ea3ecf66d2a1be"}
Sep 10 13:45:11.982504 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:11.982484    2551 scope.go:117] "RemoveContainer" containerID="2943f6752d03ae8988d7d09395d665726dd563cfec45ca2711ea3ecf66d2a1be"
Sep 10 13:45:11.983647 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:11.983612    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:45:11.983701 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:11.983691    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:45:11.983737 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:11.983727    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:45:12.990339 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:12.990303    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r" event={"ID":"67f60078-4b8c-4c34-b608-a8c065e8fee4","Type":"ContainerStarted","Data":"4ac232bab877c83f3da9642da830f7b3b13ab648c394876e676fa91059abd8c2"}
Sep 10 13:45:13.861845 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:13.861810    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:45:13.862228 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:13.862143    2551 patch_prober.go:28] interesting pod/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 10 13:45:13.862404 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:13.862381    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r" podUID="67f60078-4b8c-4c34-b608-a8c065e8fee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 10 13:45:14.862005 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:14.861965    2551 patch_prober.go:28] interesting pod/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 10 13:45:14.862503 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:14.862037    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r" podUID="67f60078-4b8c-4c34-b608-a8c065e8fee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 10 13:45:15.861852 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:15.861813    2551 patch_prober.go:28] interesting pod/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 10 13:45:15.862258 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:15.862230    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r" podUID="67f60078-4b8c-4c34-b608-a8c065e8fee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.473996    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"]
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474658    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf8bbef1-caf2-4a08-800c-bcad76e29ea2" containerName="cleanup"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474678    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf8bbef1-caf2-4a08-800c-bcad76e29ea2" containerName="cleanup"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474724    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="541c6be7-eb44-41af-b0fa-a2f5e024c8f5" containerName="main"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474733    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="541c6be7-eb44-41af-b0fa-a2f5e024c8f5" containerName="main"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474746    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="984544c4-7d1d-444f-8643-9835ca6a4aec" containerName="cleanup"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474755    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="984544c4-7d1d-444f-8643-9835ca6a4aec" containerName="cleanup"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474774    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9" containerName="main"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474782    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9" containerName="main"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474795    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7c11da9-02cd-4291-8c97-c4e012a7acb7" containerName="cleanup"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474803    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7c11da9-02cd-4291-8c97-c4e012a7acb7" containerName="cleanup"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474889    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cd19f6d-0b1e-422b-ae75-5e8ee08ae8a9" containerName="main"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474902    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf8bbef1-caf2-4a08-800c-bcad76e29ea2" containerName="cleanup"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474914    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="984544c4-7d1d-444f-8643-9835ca6a4aec" containerName="cleanup"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474922    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="541c6be7-eb44-41af-b0fa-a2f5e024c8f5" containerName="main"
Sep 10 13:45:16.475596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.474931    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7c11da9-02cd-4291-8c97-c4e012a7acb7" containerName="cleanup"
Sep 10 13:45:16.478056 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.478020    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.480300 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.480279    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-767dce\"/\"rate-test-model-767dce-kserve-self-signed-certs\""
Sep 10 13:45:16.481402 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.481373    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-767dce\"/\"kube-root-ca.crt\""
Sep 10 13:45:16.481545 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.481525    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-767dce\"/\"default-dockercfg-zvw8j\""
Sep 10 13:45:16.481626 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.481598    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-767dce\"/\"openshift-service-ca.crt\""
Sep 10 13:45:16.484291 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.484258    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"]
Sep 10 13:45:16.608136 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.608101    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5db10590-49ef-4528-90b9-c55a9d28affc-tls-certs\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.608136 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.608136    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-dshm\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.608379 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.608181    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnjnx\" (UniqueName: \"kubernetes.io/projected/5db10590-49ef-4528-90b9-c55a9d28affc-kube-api-access-hnjnx\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.608379 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.608211    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-model-cache\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.608379 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.608236    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-tmp-dir\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.608379 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.608321    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-home\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.709724 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.709670    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5db10590-49ef-4528-90b9-c55a9d28affc-tls-certs\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.709724 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.709707    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-dshm\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.709973 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.709751    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnjnx\" (UniqueName: \"kubernetes.io/projected/5db10590-49ef-4528-90b9-c55a9d28affc-kube-api-access-hnjnx\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.709973 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.709771    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-model-cache\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.709973 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.709796    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-tmp-dir\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.710124 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.710023    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-home\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.710199 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.710145    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-tmp-dir\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.710199 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.710188    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-model-cache\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.710376 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.710359    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-home\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.712104 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.712081    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-dshm\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.712207 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.712165    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5db10590-49ef-4528-90b9-c55a9d28affc-tls-certs\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.721027 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.720999    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnjnx\" (UniqueName: \"kubernetes.io/projected/5db10590-49ef-4528-90b9-c55a9d28affc-kube-api-access-hnjnx\") pod \"rate-test-model-767dce-kserve-f586ffb8c-lm6v4\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") " pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.788431 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.788355    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:16.877604 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.876272    2551 patch_prober.go:28] interesting pod/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 10 13:45:16.877604 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:16.876333    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r" podUID="67f60078-4b8c-4c34-b608-a8c065e8fee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 10 13:45:17.062651 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.060051    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"]
Sep 10 13:45:17.479400 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.479368    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"]
Sep 10 13:45:17.481842 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.481822    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.484495 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.484464    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-02a403-openshift-default-dockercfg-847bq\""
Sep 10 13:45:17.499923 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.499897    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"]
Sep 10 13:45:17.617543 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.617509    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istiod-ca-cert\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.617723 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.617592    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-workload-certs\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.617723 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.617625    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-credential-socket\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.617723 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.617653    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-envoy\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.617723 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.617699    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-token\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.617723 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.617717    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf894\" (UniqueName: \"kubernetes.io/projected/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-kube-api-access-rf894\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.617979 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.617740    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-podinfo\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.617979 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.617822    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-workload-socket\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.617979 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.617881    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-data\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.719099 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.719060    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-credential-socket\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.719286 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.719122    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-envoy\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.719286 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.719170    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-token\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.719286 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.719200    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf894\" (UniqueName: \"kubernetes.io/projected/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-kube-api-access-rf894\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.719286 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.719225    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-podinfo\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.719286 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.719266    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-workload-socket\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.719596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.719310    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-data\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.719596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.719353    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istiod-ca-cert\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.719596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.719452    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-workload-certs\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.719596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.719504    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-credential-socket\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.719972 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.719872    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-workload-socket\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.719972 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.719954    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-workload-certs\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.720091 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.719979    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-data\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.720418 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.720395    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istiod-ca-cert\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.721922 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.721900    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-envoy\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.722194 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.722176    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-podinfo\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.727654 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.727622    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-token\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.728300 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.728277    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf894\" (UniqueName: \"kubernetes.io/projected/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-kube-api-access-rf894\") pod \"e2e-api-b-02a403-openshift-default-5998c866d6-qbctv\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") " pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.794268 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.793972    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:17.867936 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.865202    2551 patch_prober.go:28] interesting pod/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 10 13:45:17.867936 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:17.865259    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r" podUID="67f60078-4b8c-4c34-b608-a8c065e8fee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 10 13:45:18.019102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:18.019063    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4" event={"ID":"5db10590-49ef-4528-90b9-c55a9d28affc","Type":"ContainerStarted","Data":"84edfc2496922fa280db7cfb8ea85582992b2dd87c163873594e0a46e3d62a82"}
Sep 10 13:45:18.019102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:18.019116    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4" event={"ID":"5db10590-49ef-4528-90b9-c55a9d28affc","Type":"ContainerStarted","Data":"0d6c3f4cafd630c03bcb77a757778e75064062aef871dd190c5114da7682edc2"}
Sep 10 13:45:18.037380 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:18.037320    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4" podStartSLOduration=2.03730091 podStartE2EDuration="2.03730091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:18.036527884 +0000 UTC m=+1628.855566285" watchObservedRunningTime="2026-09-10 13:45:18.03730091 +0000 UTC m=+1628.856339315"
Sep 10 13:45:18.071509 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:18.071484    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"]
Sep 10 13:45:18.074050 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:45:18.074025    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8efafcf_75f1_43b9_8e09_7684d5f87ad7.slice/crio-7355b8dcd1c7c6dd463a444ea826075481c2251b4405220187c45d3d9b0f2162 WatchSource:0}: Error finding container 7355b8dcd1c7c6dd463a444ea826075481c2251b4405220187c45d3d9b0f2162: Status 404 returned error can't find the container with id 7355b8dcd1c7c6dd463a444ea826075481c2251b4405220187c45d3d9b0f2162
Sep 10 13:45:18.077062 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:18.077028    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:45:18.077148 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:18.077098    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:45:18.077148 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:18.077134    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:45:18.865726 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:18.865700    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:45:18.865983 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:18.865964    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:45:18.866586 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:18.866554    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:45:19.026437 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:19.026396    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv" event={"ID":"c8efafcf-75f1-43b9-8e09-7684d5f87ad7","Type":"ContainerStarted","Data":"2adff9421bad4db12c7a92ec9b17b167dd60e877f93d3ec09067d8d08665672e"}
Sep 10 13:45:19.026899 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:19.026456    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv" event={"ID":"c8efafcf-75f1-43b9-8e09-7684d5f87ad7","Type":"ContainerStarted","Data":"7355b8dcd1c7c6dd463a444ea826075481c2251b4405220187c45d3d9b0f2162"}
Sep 10 13:45:19.049365 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:19.049314    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv" podStartSLOduration=2.049300968 podStartE2EDuration="2.049300968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:19.047496926 +0000 UTC m=+1629.866535335" watchObservedRunningTime="2026-09-10 13:45:19.049300968 +0000 UTC m=+1629.868339367"
Sep 10 13:45:19.795255 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:19.795221    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:19.800805 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:19.800776    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:20.032547 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:20.032517    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:20.033478 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:20.033461    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:20.879196 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:20.879161    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:45:26.789445 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:26.789415    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:26.789883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:26.789457    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:26.801388 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:26.801364    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:27.078692 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:27.078620    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:45:36.073767 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:36.073727    2551 patch_prober.go:28] interesting pod/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 13:45:36.074172 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:36.073815    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv" podUID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 13:45:38.129677 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:38.129606    2551 generic.go:356] "Generic (PLEG): container finished" podID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerID="2adff9421bad4db12c7a92ec9b17b167dd60e877f93d3ec09067d8d08665672e" exitCode=0
Sep 10 13:45:38.130036 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:38.129673    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv" event={"ID":"c8efafcf-75f1-43b9-8e09-7684d5f87ad7","Type":"ContainerDied","Data":"2adff9421bad4db12c7a92ec9b17b167dd60e877f93d3ec09067d8d08665672e"}
Sep 10 13:45:38.130080 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:38.130050    2551 scope.go:117] "RemoveContainer" containerID="2adff9421bad4db12c7a92ec9b17b167dd60e877f93d3ec09067d8d08665672e"
Sep 10 13:45:38.131140 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:38.131079    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:45:38.131207 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:38.131146    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:45:38.131207 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:38.131178    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:45:38.680079 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:38.680044    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:45:39.136550 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:39.136515    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv" event={"ID":"c8efafcf-75f1-43b9-8e09-7684d5f87ad7","Type":"ContainerStarted","Data":"551730c313f8670beaf81f4912d1cc12da001ab3080d7737a6bf7140b098abf8"}
Sep 10 13:45:39.794879 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:39.794841    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:39.795046 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:39.795020    2551 patch_prober.go:28] interesting pod/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 10 13:45:39.795101 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:39.795079    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv" podUID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 10 13:45:40.794938 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:40.794902    2551 patch_prober.go:28] interesting pod/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 10 13:45:40.795311 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:40.794958    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv" podUID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 10 13:45:41.795641 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:41.795598    2551 patch_prober.go:28] interesting pod/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 10 13:45:41.796026 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:41.795670    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv" podUID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 10 13:45:42.795988 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:42.795945    2551 patch_prober.go:28] interesting pod/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 10 13:45:42.796476 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:42.796010    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv" podUID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 10 13:45:43.795718 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:43.795686    2551 patch_prober.go:28] interesting pod/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 10 13:45:43.795892 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:43.795737    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv" podUID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 10 13:45:44.799096 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:44.799061    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:44.799511 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:44.799332    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:44.800172 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:44.800157    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:45:48.951058 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:48.951026    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn"]
Sep 10 13:45:48.954332 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:48.954314    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn"
Sep 10 13:45:48.957165 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:48.957144    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:45:48.957359 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:48.957341    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:45:48.964011 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:48.963984    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn"]
Sep 10 13:45:49.027621 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:49.027564    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f674d22-f808-4cf7-8587-b070ae94eb90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn\" (UID: \"7f674d22-f808-4cf7-8587-b070ae94eb90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn"
Sep 10 13:45:49.128325 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:49.128289    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f674d22-f808-4cf7-8587-b070ae94eb90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn\" (UID: \"7f674d22-f808-4cf7-8587-b070ae94eb90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn"
Sep 10 13:45:49.128909 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:49.128890    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f674d22-f808-4cf7-8587-b070ae94eb90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn\" (UID: \"7f674d22-f808-4cf7-8587-b070ae94eb90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn"
Sep 10 13:45:49.266603 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:49.266563    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn"
Sep 10 13:45:49.538929 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:49.538900    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn"]
Sep 10 13:45:49.540278 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:45:49.540253    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f674d22_f808_4cf7_8587_b070ae94eb90.slice/crio-58d173290b3716e51fccccdc9fcdf726ca092e9cd0f4f01013790df390fa40d7 WatchSource:0}: Error finding container 58d173290b3716e51fccccdc9fcdf726ca092e9cd0f4f01013790df390fa40d7: Status 404 returned error can't find the container with id 58d173290b3716e51fccccdc9fcdf726ca092e9cd0f4f01013790df390fa40d7
Sep 10 13:45:50.205744 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:50.205708    2551 generic.go:356] "Generic (PLEG): container finished" podID="7f674d22-f808-4cf7-8587-b070ae94eb90" containerID="0fb06c256335107c250b7ecf04e8e52e82e99343620d133605ae1a3d5c2fbc13" exitCode=0
Sep 10 13:45:50.206082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:50.205785    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn" event={"ID":"7f674d22-f808-4cf7-8587-b070ae94eb90","Type":"ContainerDied","Data":"0fb06c256335107c250b7ecf04e8e52e82e99343620d133605ae1a3d5c2fbc13"}
Sep 10 13:45:50.206082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:50.205860    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn" event={"ID":"7f674d22-f808-4cf7-8587-b070ae94eb90","Type":"ContainerStarted","Data":"58d173290b3716e51fccccdc9fcdf726ca092e9cd0f4f01013790df390fa40d7"}
Sep 10 13:45:51.337079 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:51.337056    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn"
Sep 10 13:45:51.449282 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:51.449252    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f674d22-f808-4cf7-8587-b070ae94eb90-maas-api-service-ca\") pod \"7f674d22-f808-4cf7-8587-b070ae94eb90\" (UID: \"7f674d22-f808-4cf7-8587-b070ae94eb90\") "
Sep 10 13:45:51.449661 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:51.449640    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f674d22-f808-4cf7-8587-b070ae94eb90-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7f674d22-f808-4cf7-8587-b070ae94eb90" (UID: "7f674d22-f808-4cf7-8587-b070ae94eb90"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:45:51.550823 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:51.550800    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f674d22-f808-4cf7-8587-b070ae94eb90-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:45:52.217264 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:52.217228    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn" event={"ID":"7f674d22-f808-4cf7-8587-b070ae94eb90","Type":"ContainerDied","Data":"58d173290b3716e51fccccdc9fcdf726ca092e9cd0f4f01013790df390fa40d7"}
Sep 10 13:45:52.217264 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:52.217254    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn"
Sep 10 13:45:52.217476 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:45:52.217281    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58d173290b3716e51fccccdc9fcdf726ca092e9cd0f4f01013790df390fa40d7"
Sep 10 13:46:03.764162 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.764131    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"]
Sep 10 13:46:03.764526 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.764486    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f674d22-f808-4cf7-8587-b070ae94eb90" containerName="revoke-keys"
Sep 10 13:46:03.764526 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.764496    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f674d22-f808-4cf7-8587-b070ae94eb90" containerName="revoke-keys"
Sep 10 13:46:03.764625 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.764553    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f674d22-f808-4cf7-8587-b070ae94eb90" containerName="revoke-keys"
Sep 10 13:46:03.767605 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.767564    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.771324 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.771302    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-89be92\"/\"openshift-service-ca.crt\""
Sep 10 13:46:03.771414 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.771352    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-89be92\"/\"rate-test-model-89be92-kserve-self-signed-certs\""
Sep 10 13:46:03.771414 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.771376    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-89be92\"/\"default-dockercfg-2ldvq\""
Sep 10 13:46:03.771523 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.771507    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-89be92\"/\"kube-root-ca.crt\""
Sep 10 13:46:03.778700 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.778682    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"]
Sep 10 13:46:03.859338 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.859311    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9wvg\" (UniqueName: \"kubernetes.io/projected/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-kube-api-access-q9wvg\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.859460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.859347    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-model-cache\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.859460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.859375    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-tmp-dir\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.859460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.859391    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-home\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.859644 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.859464    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-tls-certs\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.859644 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.859566    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-dshm\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.960102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.960077    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-dshm\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.960209 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.960128    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9wvg\" (UniqueName: \"kubernetes.io/projected/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-kube-api-access-q9wvg\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.960209 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.960155    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-model-cache\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.960209 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.960182    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-tmp-dir\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.960209 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.960198    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-home\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.960416 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.960217    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-tls-certs\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.960560 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.960538    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-tmp-dir\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.960700 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.960557    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-model-cache\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.960700 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.960619    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-home\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.962204 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.962179    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-dshm\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.962432 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.962416    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-tls-certs\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:03.969886 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:03.969859    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9wvg\" (UniqueName: \"kubernetes.io/projected/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-kube-api-access-q9wvg\") pod \"rate-test-model-89be92-kserve-76db94d467-qcj96\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") " pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:04.076259 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:04.076161    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:04.348990 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:04.348965    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"]
Sep 10 13:46:04.350557 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:46:04.350531    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod460358b1_9ba3_48e3_8e8b_92a2b0c1d954.slice/crio-21f6b2265966b1171a7cf345c1845f1caef8236e52b12667a4ea207190bb35ac WatchSource:0}: Error finding container 21f6b2265966b1171a7cf345c1845f1caef8236e52b12667a4ea207190bb35ac: Status 404 returned error can't find the container with id 21f6b2265966b1171a7cf345c1845f1caef8236e52b12667a4ea207190bb35ac
Sep 10 13:46:05.033725 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:05.033692    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn"]
Sep 10 13:46:05.037202 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:05.037178    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2d945-9sbtn"]
Sep 10 13:46:05.282847 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:05.282810    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96" event={"ID":"460358b1-9ba3-48e3-8e8b-92a2b0c1d954","Type":"ContainerStarted","Data":"de65da63cf03516455341ee9df1503cbd96272ad36a37ef5ae0b5872c84a3424"}
Sep 10 13:46:05.282847 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:05.282851    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96" event={"ID":"460358b1-9ba3-48e3-8e8b-92a2b0c1d954","Type":"ContainerStarted","Data":"21f6b2265966b1171a7cf345c1845f1caef8236e52b12667a4ea207190bb35ac"}
Sep 10 13:46:05.304267 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:05.304182    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96" podStartSLOduration=2.30416911 podStartE2EDuration="2.30416911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:46:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:46:05.303384121 +0000 UTC m=+1676.122422521" watchObservedRunningTime="2026-09-10 13:46:05.30416911 +0000 UTC m=+1676.123207491"
Sep 10 13:46:05.710200 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:05.710123    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f674d22-f808-4cf7-8587-b070ae94eb90" path="/var/lib/kubelet/pods/7f674d22-f808-4cf7-8587-b070ae94eb90/volumes"
Sep 10 13:46:09.084925 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:09.084889    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:46:09.925893 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:09.925860    2551 scope.go:117] "RemoveContainer" containerID="236f91cca050249b266a1bd275b7a8ee1a5f6bc083087b10cac35fe80fd1a43e"
Sep 10 13:46:12.141380 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:12.141344    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"]
Sep 10 13:46:12.141889 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:12.141551    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r" podUID="67f60078-4b8c-4c34-b608-a8c065e8fee4" containerName="istio-proxy" containerID="cri-o://4ac232bab877c83f3da9642da830f7b3b13ab648c394876e676fa91059abd8c2" gracePeriod=30
Sep 10 13:46:12.521733 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:12.521702    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws"]
Sep 10 13:46:12.525235 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:12.525217    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws"
Sep 10 13:46:12.528867 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:12.528846    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:46:12.528995 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:12.528897    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:46:12.536931 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:12.536893    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws"]
Sep 10 13:46:12.650989 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:12.650956    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/922144d8-2e96-4982-bc45-0a0c9eb4134f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-02a403-stwws\" (UID: \"922144d8-2e96-4982-bc45-0a0c9eb4134f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws"
Sep 10 13:46:12.751722 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:12.751689    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/922144d8-2e96-4982-bc45-0a0c9eb4134f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-02a403-stwws\" (UID: \"922144d8-2e96-4982-bc45-0a0c9eb4134f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws"
Sep 10 13:46:12.752319 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:12.752298    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/922144d8-2e96-4982-bc45-0a0c9eb4134f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-02a403-stwws\" (UID: \"922144d8-2e96-4982-bc45-0a0c9eb4134f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws"
Sep 10 13:46:12.836065 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:12.835979    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws"
Sep 10 13:46:13.109933 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:13.109904    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws"]
Sep 10 13:46:13.110494 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:46:13.110474    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922144d8_2e96_4982_bc45_0a0c9eb4134f.slice/crio-08169e2f7d7780ff77e12925371d0f089a27944b47a957916f4fd877034cd038 WatchSource:0}: Error finding container 08169e2f7d7780ff77e12925371d0f089a27944b47a957916f4fd877034cd038: Status 404 returned error can't find the container with id 08169e2f7d7780ff77e12925371d0f089a27944b47a957916f4fd877034cd038
Sep 10 13:46:13.334249 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:13.334211    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws" event={"ID":"922144d8-2e96-4982-bc45-0a0c9eb4134f","Type":"ContainerStarted","Data":"08169e2f7d7780ff77e12925371d0f089a27944b47a957916f4fd877034cd038"}
Sep 10 13:46:14.076424 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:14.076378    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:14.076626 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:14.076437    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:14.089158 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:14.089124    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:14.341026 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:14.340984    2551 generic.go:356] "Generic (PLEG): container finished" podID="922144d8-2e96-4982-bc45-0a0c9eb4134f" containerID="af0c0438fed846fefbfc4719c2c108b3c8ab7d05cd2923982cab6ae4b72a8903" exitCode=0
Sep 10 13:46:14.341503 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:14.341053    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws" event={"ID":"922144d8-2e96-4982-bc45-0a0c9eb4134f","Type":"ContainerDied","Data":"af0c0438fed846fefbfc4719c2c108b3c8ab7d05cd2923982cab6ae4b72a8903"}
Sep 10 13:46:14.352252 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:14.352229    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:46:15.479224 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:15.479197    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws"
Sep 10 13:46:15.580567 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:15.580534    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/922144d8-2e96-4982-bc45-0a0c9eb4134f-maas-api-service-ca\") pod \"922144d8-2e96-4982-bc45-0a0c9eb4134f\" (UID: \"922144d8-2e96-4982-bc45-0a0c9eb4134f\") "
Sep 10 13:46:15.580928 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:15.580904    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/922144d8-2e96-4982-bc45-0a0c9eb4134f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "922144d8-2e96-4982-bc45-0a0c9eb4134f" (UID: "922144d8-2e96-4982-bc45-0a0c9eb4134f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:46:15.581023 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:15.581010    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/922144d8-2e96-4982-bc45-0a0c9eb4134f-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:16.352849 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:16.352820    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws"
Sep 10 13:46:16.352849 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:16.352836    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws" event={"ID":"922144d8-2e96-4982-bc45-0a0c9eb4134f","Type":"ContainerDied","Data":"08169e2f7d7780ff77e12925371d0f089a27944b47a957916f4fd877034cd038"}
Sep 10 13:46:16.353052 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:16.352880    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08169e2f7d7780ff77e12925371d0f089a27944b47a957916f4fd877034cd038"
Sep 10 13:46:17.361641 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.361611    2551 generic.go:356] "Generic (PLEG): container finished" podID="67f60078-4b8c-4c34-b608-a8c065e8fee4" containerID="4ac232bab877c83f3da9642da830f7b3b13ab648c394876e676fa91059abd8c2" exitCode=0
Sep 10 13:46:17.362007 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.361683    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r" event={"ID":"67f60078-4b8c-4c34-b608-a8c065e8fee4","Type":"ContainerDied","Data":"4ac232bab877c83f3da9642da830f7b3b13ab648c394876e676fa91059abd8c2"}
Sep 10 13:46:17.362007 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.361750    2551 scope.go:117] "RemoveContainer" containerID="2943f6752d03ae8988d7d09395d665726dd563cfec45ca2711ea3ecf66d2a1be"
Sep 10 13:46:17.398758 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.398731    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:46:17.499330 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.499299    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-token\") pod \"67f60078-4b8c-4c34-b608-a8c065e8fee4\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") "
Sep 10 13:46:17.499497 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.499345    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-credential-socket\") pod \"67f60078-4b8c-4c34-b608-a8c065e8fee4\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") "
Sep 10 13:46:17.499497 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.499370    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-workload-socket\") pod \"67f60078-4b8c-4c34-b608-a8c065e8fee4\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") "
Sep 10 13:46:17.499497 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.499389    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67f60078-4b8c-4c34-b608-a8c065e8fee4-istiod-ca-cert\") pod \"67f60078-4b8c-4c34-b608-a8c065e8fee4\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") "
Sep 10 13:46:17.499497 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.499426    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-data\") pod \"67f60078-4b8c-4c34-b608-a8c065e8fee4\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") "
Sep 10 13:46:17.499497 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.499457    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-workload-certs\") pod \"67f60078-4b8c-4c34-b608-a8c065e8fee4\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") "
Sep 10 13:46:17.499827 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.499537    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjkhd\" (UniqueName: \"kubernetes.io/projected/67f60078-4b8c-4c34-b608-a8c065e8fee4-kube-api-access-hjkhd\") pod \"67f60078-4b8c-4c34-b608-a8c065e8fee4\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") "
Sep 10 13:46:17.499827 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.499600    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-envoy\") pod \"67f60078-4b8c-4c34-b608-a8c065e8fee4\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") "
Sep 10 13:46:17.499827 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.499672    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "67f60078-4b8c-4c34-b608-a8c065e8fee4" (UID: "67f60078-4b8c-4c34-b608-a8c065e8fee4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:17.499827 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.499702    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-podinfo\") pod \"67f60078-4b8c-4c34-b608-a8c065e8fee4\" (UID: \"67f60078-4b8c-4c34-b608-a8c065e8fee4\") "
Sep 10 13:46:17.499827 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.499710    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "67f60078-4b8c-4c34-b608-a8c065e8fee4" (UID: "67f60078-4b8c-4c34-b608-a8c065e8fee4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:17.500082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.499827    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "67f60078-4b8c-4c34-b608-a8c065e8fee4" (UID: "67f60078-4b8c-4c34-b608-a8c065e8fee4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:17.500082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.499961    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-data" (OuterVolumeSpecName: "istio-data") pod "67f60078-4b8c-4c34-b608-a8c065e8fee4" (UID: "67f60078-4b8c-4c34-b608-a8c065e8fee4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:17.500198 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.500081    2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-credential-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:17.500198 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.500099    2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-workload-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:17.500198 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.500114    2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-data\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:17.500198 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.500126    2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-workload-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:17.500359 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.500263    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67f60078-4b8c-4c34-b608-a8c065e8fee4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "67f60078-4b8c-4c34-b608-a8c065e8fee4" (UID: "67f60078-4b8c-4c34-b608-a8c065e8fee4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:46:17.502346 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.502314    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "67f60078-4b8c-4c34-b608-a8c065e8fee4" (UID: "67f60078-4b8c-4c34-b608-a8c065e8fee4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:17.502453 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.502346    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67f60078-4b8c-4c34-b608-a8c065e8fee4-kube-api-access-hjkhd" (OuterVolumeSpecName: "kube-api-access-hjkhd") pod "67f60078-4b8c-4c34-b608-a8c065e8fee4" (UID: "67f60078-4b8c-4c34-b608-a8c065e8fee4"). InnerVolumeSpecName "kube-api-access-hjkhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:46:17.502453 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.502384    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-token" (OuterVolumeSpecName: "istio-token") pod "67f60078-4b8c-4c34-b608-a8c065e8fee4" (UID: "67f60078-4b8c-4c34-b608-a8c065e8fee4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:46:17.502453 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.502405    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "67f60078-4b8c-4c34-b608-a8c065e8fee4" (UID: "67f60078-4b8c-4c34-b608-a8c065e8fee4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 13:46:17.601589 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.601539    2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67f60078-4b8c-4c34-b608-a8c065e8fee4-istiod-ca-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:17.601589 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.601603    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjkhd\" (UniqueName: \"kubernetes.io/projected/67f60078-4b8c-4c34-b608-a8c065e8fee4-kube-api-access-hjkhd\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:17.601810 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.601616    2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-envoy\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:17.601810 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.601625    2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-podinfo\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:17.601810 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:17.601634    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67f60078-4b8c-4c34-b608-a8c065e8fee4-istio-token\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:18.368483 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:18.368441    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r" event={"ID":"67f60078-4b8c-4c34-b608-a8c065e8fee4","Type":"ContainerDied","Data":"3c2daf227db3c5b1e8d722e04ae0a79529dce967ab1e7102e9203f81766a7ba6"}
Sep 10 13:46:18.368483 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:18.368480    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"
Sep 10 13:46:18.369025 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:18.368510    2551 scope.go:117] "RemoveContainer" containerID="4ac232bab877c83f3da9642da830f7b3b13ab648c394876e676fa91059abd8c2"
Sep 10 13:46:18.394249 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:18.394218    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"]
Sep 10 13:46:18.401372 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:18.401349    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a2d945-openshift-default-58d5fcbc98-f2m7r"]
Sep 10 13:46:19.710004 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:19.709972    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67f60078-4b8c-4c34-b608-a8c065e8fee4" path="/var/lib/kubelet/pods/67f60078-4b8c-4c34-b608-a8c065e8fee4/volumes"
Sep 10 13:46:28.312193 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:28.312157    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws"]
Sep 10 13:46:28.315313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:28.315288    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-02a403-stwws"]
Sep 10 13:46:29.709856 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:29.709816    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="922144d8-2e96-4982-bc45-0a0c9eb4134f" path="/var/lib/kubelet/pods/922144d8-2e96-4982-bc45-0a0c9eb4134f/volumes"
Sep 10 13:46:33.263110 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:33.263037    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:46:35.279169 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.279135    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"]
Sep 10 13:46:35.279647 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.279382    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv" podUID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerName="istio-proxy" containerID="cri-o://551730c313f8670beaf81f4912d1cc12da001ab3080d7737a6bf7140b098abf8" gracePeriod=30
Sep 10 13:46:35.646101 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.646022    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9"]
Sep 10 13:46:35.646381 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.646368    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="922144d8-2e96-4982-bc45-0a0c9eb4134f" containerName="revoke-keys"
Sep 10 13:46:35.646425 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.646382    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="922144d8-2e96-4982-bc45-0a0c9eb4134f" containerName="revoke-keys"
Sep 10 13:46:35.646425 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.646406    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67f60078-4b8c-4c34-b608-a8c065e8fee4" containerName="istio-proxy"
Sep 10 13:46:35.646425 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.646413    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="67f60078-4b8c-4c34-b608-a8c065e8fee4" containerName="istio-proxy"
Sep 10 13:46:35.646425 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.646422    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67f60078-4b8c-4c34-b608-a8c065e8fee4" containerName="istio-proxy"
Sep 10 13:46:35.646548 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.646428    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="67f60078-4b8c-4c34-b608-a8c065e8fee4" containerName="istio-proxy"
Sep 10 13:46:35.646548 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.646487    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="922144d8-2e96-4982-bc45-0a0c9eb4134f" containerName="revoke-keys"
Sep 10 13:46:35.646548 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.646498    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="67f60078-4b8c-4c34-b608-a8c065e8fee4" containerName="istio-proxy"
Sep 10 13:46:35.646548 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.646504    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="67f60078-4b8c-4c34-b608-a8c065e8fee4" containerName="istio-proxy"
Sep 10 13:46:35.649967 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.649949    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9"
Sep 10 13:46:35.652565 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.652547    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:46:35.652649 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.652547    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:46:35.657695 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.657673    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9"]
Sep 10 13:46:35.771056 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.771022    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6127a1-7340-420c-a589-e4f0f6196641-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9\" (UID: \"2c6127a1-7340-420c-a589-e4f0f6196641\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9"
Sep 10 13:46:35.871946 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.871910    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6127a1-7340-420c-a589-e4f0f6196641-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9\" (UID: \"2c6127a1-7340-420c-a589-e4f0f6196641\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9"
Sep 10 13:46:35.872507 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.872486    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6127a1-7340-420c-a589-e4f0f6196641-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9\" (UID: \"2c6127a1-7340-420c-a589-e4f0f6196641\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9"
Sep 10 13:46:35.961643 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:35.961552    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9"
Sep 10 13:46:36.232454 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:36.232430    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9"]
Sep 10 13:46:36.234212 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:46:36.234189    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c6127a1_7340_420c_a589_e4f0f6196641.slice/crio-b2c2b6c28921b02c9594e8603419b367941eca93f6a0baed9feb9d58f5f57cff WatchSource:0}: Error finding container b2c2b6c28921b02c9594e8603419b367941eca93f6a0baed9feb9d58f5f57cff: Status 404 returned error can't find the container with id b2c2b6c28921b02c9594e8603419b367941eca93f6a0baed9feb9d58f5f57cff
Sep 10 13:46:36.464835 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:36.464802    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9" event={"ID":"2c6127a1-7340-420c-a589-e4f0f6196641","Type":"ContainerStarted","Data":"b2c2b6c28921b02c9594e8603419b367941eca93f6a0baed9feb9d58f5f57cff"}
Sep 10 13:46:37.473924 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:37.473889    2551 generic.go:356] "Generic (PLEG): container finished" podID="2c6127a1-7340-420c-a589-e4f0f6196641" containerID="46e06c2828a0c8fa06583649718c68a8b65eae0d407ccaaa0be5cb28e2e54a4e" exitCode=0
Sep 10 13:46:37.474302 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:37.473947    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9" event={"ID":"2c6127a1-7340-420c-a589-e4f0f6196641","Type":"ContainerDied","Data":"46e06c2828a0c8fa06583649718c68a8b65eae0d407ccaaa0be5cb28e2e54a4e"}
Sep 10 13:46:38.614037 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:38.614012    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9"
Sep 10 13:46:38.696847 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:38.696819    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6127a1-7340-420c-a589-e4f0f6196641-maas-api-service-ca\") pod \"2c6127a1-7340-420c-a589-e4f0f6196641\" (UID: \"2c6127a1-7340-420c-a589-e4f0f6196641\") "
Sep 10 13:46:38.697158 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:38.697133    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c6127a1-7340-420c-a589-e4f0f6196641-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2c6127a1-7340-420c-a589-e4f0f6196641" (UID: "2c6127a1-7340-420c-a589-e4f0f6196641"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:46:38.798458 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:38.798375    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6127a1-7340-420c-a589-e4f0f6196641-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:39.487028 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:39.486996    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9"
Sep 10 13:46:39.487192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:39.486998    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9" event={"ID":"2c6127a1-7340-420c-a589-e4f0f6196641","Type":"ContainerDied","Data":"b2c2b6c28921b02c9594e8603419b367941eca93f6a0baed9feb9d58f5f57cff"}
Sep 10 13:46:39.487192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:39.487130    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2c2b6c28921b02c9594e8603419b367941eca93f6a0baed9feb9d58f5f57cff"
Sep 10 13:46:40.494846 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.494813    2551 generic.go:356] "Generic (PLEG): container finished" podID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerID="551730c313f8670beaf81f4912d1cc12da001ab3080d7737a6bf7140b098abf8" exitCode=0
Sep 10 13:46:40.495200 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.494888    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv" event={"ID":"c8efafcf-75f1-43b9-8e09-7684d5f87ad7","Type":"ContainerDied","Data":"551730c313f8670beaf81f4912d1cc12da001ab3080d7737a6bf7140b098abf8"}
Sep 10 13:46:40.495200 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.494961    2551 scope.go:117] "RemoveContainer" containerID="2adff9421bad4db12c7a92ec9b17b167dd60e877f93d3ec09067d8d08665672e"
Sep 10 13:46:40.528924 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.528898    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:46:40.613636 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.613554    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istiod-ca-cert\") pod \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") "
Sep 10 13:46:40.613785 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.613630    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-data\") pod \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") "
Sep 10 13:46:40.613785 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.613702    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-credential-socket\") pod \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") "
Sep 10 13:46:40.613785 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.613733    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-workload-certs\") pod \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") "
Sep 10 13:46:40.613785 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.613762    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-workload-socket\") pod \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") "
Sep 10 13:46:40.614013 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.613786    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-envoy\") pod \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") "
Sep 10 13:46:40.614013 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.613817    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-token\") pod \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") "
Sep 10 13:46:40.614013 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.613854    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-podinfo\") pod \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") "
Sep 10 13:46:40.614013 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.613914    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf894\" (UniqueName: \"kubernetes.io/projected/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-kube-api-access-rf894\") pod \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\" (UID: \"c8efafcf-75f1-43b9-8e09-7684d5f87ad7\") "
Sep 10 13:46:40.614013 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.613947    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-data" (OuterVolumeSpecName: "istio-data") pod "c8efafcf-75f1-43b9-8e09-7684d5f87ad7" (UID: "c8efafcf-75f1-43b9-8e09-7684d5f87ad7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:40.614336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.614039    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c8efafcf-75f1-43b9-8e09-7684d5f87ad7" (UID: "c8efafcf-75f1-43b9-8e09-7684d5f87ad7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:40.614336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.614059    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c8efafcf-75f1-43b9-8e09-7684d5f87ad7" (UID: "c8efafcf-75f1-43b9-8e09-7684d5f87ad7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:40.614336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.614071    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c8efafcf-75f1-43b9-8e09-7684d5f87ad7" (UID: "c8efafcf-75f1-43b9-8e09-7684d5f87ad7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:46:40.614336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.614072    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c8efafcf-75f1-43b9-8e09-7684d5f87ad7" (UID: "c8efafcf-75f1-43b9-8e09-7684d5f87ad7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:40.614336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.614297    2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-credential-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:40.614336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.614319    2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-workload-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:40.614336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.614334    2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-workload-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:40.614668 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.614350    2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istiod-ca-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:40.614668 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.614365    2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-data\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:40.616095 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.616075    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c8efafcf-75f1-43b9-8e09-7684d5f87ad7" (UID: "c8efafcf-75f1-43b9-8e09-7684d5f87ad7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:40.616095 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.616084    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c8efafcf-75f1-43b9-8e09-7684d5f87ad7" (UID: "c8efafcf-75f1-43b9-8e09-7684d5f87ad7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 13:46:40.616383 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.616361    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-token" (OuterVolumeSpecName: "istio-token") pod "c8efafcf-75f1-43b9-8e09-7684d5f87ad7" (UID: "c8efafcf-75f1-43b9-8e09-7684d5f87ad7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:46:40.616451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.616378    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-kube-api-access-rf894" (OuterVolumeSpecName: "kube-api-access-rf894") pod "c8efafcf-75f1-43b9-8e09-7684d5f87ad7" (UID: "c8efafcf-75f1-43b9-8e09-7684d5f87ad7"). InnerVolumeSpecName "kube-api-access-rf894". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:46:40.715304 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.715264    2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-envoy\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:40.715304 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.715298    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-token\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:40.715304 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.715309    2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-istio-podinfo\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:40.715535 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:40.715320    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf894\" (UniqueName: \"kubernetes.io/projected/c8efafcf-75f1-43b9-8e09-7684d5f87ad7-kube-api-access-rf894\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:41.501782 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:41.501752    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"
Sep 10 13:46:41.502178 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:41.501749    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv" event={"ID":"c8efafcf-75f1-43b9-8e09-7684d5f87ad7","Type":"ContainerDied","Data":"7355b8dcd1c7c6dd463a444ea826075481c2251b4405220187c45d3d9b0f2162"}
Sep 10 13:46:41.502178 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:41.501892    2551 scope.go:117] "RemoveContainer" containerID="551730c313f8670beaf81f4912d1cc12da001ab3080d7737a6bf7140b098abf8"
Sep 10 13:46:41.524604 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:41.524556    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"]
Sep 10 13:46:41.526611 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:41.526586    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-02a403-openshift-default-5998c866d6-qbctv"]
Sep 10 13:46:41.709974 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:41.709932    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" path="/var/lib/kubelet/pods/c8efafcf-75f1-43b9-8e09-7684d5f87ad7/volumes"
Sep 10 13:46:46.384724 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:46.384689    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9"]
Sep 10 13:46:46.387538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:46.387516    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5d75ac-8w4d9"]
Sep 10 13:46:47.710018 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:47.709983    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c6127a1-7340-420c-a589-e4f0f6196641" path="/var/lib/kubelet/pods/2c6127a1-7340-420c-a589-e4f0f6196641/volumes"
Sep 10 13:46:51.269235 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:51.269198    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:46:53.502040 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:53.502006    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"]
Sep 10 13:46:53.502444 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:53.502226    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f" podUID="280f3887-9106-434c-8088-5086ee526ed0" containerName="istio-proxy" containerID="cri-o://5e49fd0973a3574729b761487f066a1fd847b4352d4adf3d46fa7fb6c1ae370b" gracePeriod=30
Sep 10 13:46:53.849007 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:53.848934    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb"]
Sep 10 13:46:53.849280 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:53.849268    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerName="istio-proxy"
Sep 10 13:46:53.849321 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:53.849281    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerName="istio-proxy"
Sep 10 13:46:53.849321 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:53.849290    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c6127a1-7340-420c-a589-e4f0f6196641" containerName="revoke-keys"
Sep 10 13:46:53.849321 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:53.849296    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c6127a1-7340-420c-a589-e4f0f6196641" containerName="revoke-keys"
Sep 10 13:46:53.849422 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:53.849369    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c6127a1-7340-420c-a589-e4f0f6196641" containerName="revoke-keys"
Sep 10 13:46:53.849422 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:53.849379    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerName="istio-proxy"
Sep 10 13:46:53.853774 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:53.853758    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb"
Sep 10 13:46:53.856101 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:53.856078    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:46:53.856220 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:53.856133    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:46:53.860554 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:53.860530    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb"]
Sep 10 13:46:53.941535 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:53.941501    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32a5594-8e5a-4955-8e40-1b552f2e3fff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb\" (UID: \"f32a5594-8e5a-4955-8e40-1b552f2e3fff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb"
Sep 10 13:46:54.042955 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:54.042928    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32a5594-8e5a-4955-8e40-1b552f2e3fff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb\" (UID: \"f32a5594-8e5a-4955-8e40-1b552f2e3fff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb"
Sep 10 13:46:54.043523 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:54.043503    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32a5594-8e5a-4955-8e40-1b552f2e3fff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb\" (UID: \"f32a5594-8e5a-4955-8e40-1b552f2e3fff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb"
Sep 10 13:46:54.165241 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:54.165146    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb"
Sep 10 13:46:54.432555 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:54.432525    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb"]
Sep 10 13:46:54.434007 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:46:54.433982    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32a5594_8e5a_4955_8e40_1b552f2e3fff.slice/crio-881d7edbdfa218e8f48ade3e2b0f4ddfccf69bf9a16ee4a8f87b93c0b4823a83 WatchSource:0}: Error finding container 881d7edbdfa218e8f48ade3e2b0f4ddfccf69bf9a16ee4a8f87b93c0b4823a83: Status 404 returned error can't find the container with id 881d7edbdfa218e8f48ade3e2b0f4ddfccf69bf9a16ee4a8f87b93c0b4823a83
Sep 10 13:46:54.571831 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:54.571797    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb" event={"ID":"f32a5594-8e5a-4955-8e40-1b552f2e3fff","Type":"ContainerStarted","Data":"881d7edbdfa218e8f48ade3e2b0f4ddfccf69bf9a16ee4a8f87b93c0b4823a83"}
Sep 10 13:46:55.577841 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:55.577804    2551 generic.go:356] "Generic (PLEG): container finished" podID="f32a5594-8e5a-4955-8e40-1b552f2e3fff" containerID="fe618a21bbd0818af786ec19fcdef627a7a57195a79edc27057fccb3fd3f44f7" exitCode=0
Sep 10 13:46:55.578194 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:55.577895    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb" event={"ID":"f32a5594-8e5a-4955-8e40-1b552f2e3fff","Type":"ContainerDied","Data":"fe618a21bbd0818af786ec19fcdef627a7a57195a79edc27057fccb3fd3f44f7"}
Sep 10 13:46:56.707970 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:56.707943    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb"
Sep 10 13:46:56.865120 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:56.865065    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32a5594-8e5a-4955-8e40-1b552f2e3fff-maas-api-service-ca\") pod \"f32a5594-8e5a-4955-8e40-1b552f2e3fff\" (UID: \"f32a5594-8e5a-4955-8e40-1b552f2e3fff\") "
Sep 10 13:46:56.865417 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:56.865394    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f32a5594-8e5a-4955-8e40-1b552f2e3fff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f32a5594-8e5a-4955-8e40-1b552f2e3fff" (UID: "f32a5594-8e5a-4955-8e40-1b552f2e3fff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:46:56.965941 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:56.965915    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32a5594-8e5a-4955-8e40-1b552f2e3fff-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:57.526559 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:57.526525    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"]
Sep 10 13:46:57.591044 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:57.591014    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb"
Sep 10 13:46:57.591200 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:57.591063    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb" event={"ID":"f32a5594-8e5a-4955-8e40-1b552f2e3fff","Type":"ContainerDied","Data":"881d7edbdfa218e8f48ade3e2b0f4ddfccf69bf9a16ee4a8f87b93c0b4823a83"}
Sep 10 13:46:57.591200 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:57.591113    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="881d7edbdfa218e8f48ade3e2b0f4ddfccf69bf9a16ee4a8f87b93c0b4823a83"
Sep 10 13:46:58.585616 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.585584    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"]
Sep 10 13:46:58.753876 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.753853    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:46:58.882686 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.882612    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280f3887-9106-434c-8088-5086ee526ed0-istio-podinfo\") pod \"280f3887-9106-434c-8088-5086ee526ed0\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") "
Sep 10 13:46:58.882686 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.882654    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-workload-socket\") pod \"280f3887-9106-434c-8088-5086ee526ed0\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") "
Sep 10 13:46:58.882686 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.882671    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-istio-envoy\") pod \"280f3887-9106-434c-8088-5086ee526ed0\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") "
Sep 10 13:46:58.882960 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.882693    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5k87\" (UniqueName: \"kubernetes.io/projected/280f3887-9106-434c-8088-5086ee526ed0-kube-api-access-t5k87\") pod \"280f3887-9106-434c-8088-5086ee526ed0\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") "
Sep 10 13:46:58.882960 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.882736    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280f3887-9106-434c-8088-5086ee526ed0-istio-token\") pod \"280f3887-9106-434c-8088-5086ee526ed0\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") "
Sep 10 13:46:58.882960 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.882795    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-credential-socket\") pod \"280f3887-9106-434c-8088-5086ee526ed0\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") "
Sep 10 13:46:58.882960 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.882877    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280f3887-9106-434c-8088-5086ee526ed0-istiod-ca-cert\") pod \"280f3887-9106-434c-8088-5086ee526ed0\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") "
Sep 10 13:46:58.882960 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.882895    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "280f3887-9106-434c-8088-5086ee526ed0" (UID: "280f3887-9106-434c-8088-5086ee526ed0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:58.882960 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.882919    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-istio-data\") pod \"280f3887-9106-434c-8088-5086ee526ed0\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") "
Sep 10 13:46:58.882960 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.882946    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-workload-certs\") pod \"280f3887-9106-434c-8088-5086ee526ed0\" (UID: \"280f3887-9106-434c-8088-5086ee526ed0\") "
Sep 10 13:46:58.883330 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.883167    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-istio-data" (OuterVolumeSpecName: "istio-data") pod "280f3887-9106-434c-8088-5086ee526ed0" (UID: "280f3887-9106-434c-8088-5086ee526ed0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:58.883390 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.883330    2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-istio-data\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:58.883390 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.883352    2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-workload-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:58.883390 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.883365    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "280f3887-9106-434c-8088-5086ee526ed0" (UID: "280f3887-9106-434c-8088-5086ee526ed0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:58.883390 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.883367    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/280f3887-9106-434c-8088-5086ee526ed0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "280f3887-9106-434c-8088-5086ee526ed0" (UID: "280f3887-9106-434c-8088-5086ee526ed0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:46:58.883651 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.883435    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "280f3887-9106-434c-8088-5086ee526ed0" (UID: "280f3887-9106-434c-8088-5086ee526ed0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:58.884884 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.884855    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/280f3887-9106-434c-8088-5086ee526ed0-istio-token" (OuterVolumeSpecName: "istio-token") pod "280f3887-9106-434c-8088-5086ee526ed0" (UID: "280f3887-9106-434c-8088-5086ee526ed0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:46:58.885001 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.884962    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/280f3887-9106-434c-8088-5086ee526ed0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "280f3887-9106-434c-8088-5086ee526ed0" (UID: "280f3887-9106-434c-8088-5086ee526ed0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 13:46:58.885058 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.884995    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/280f3887-9106-434c-8088-5086ee526ed0-kube-api-access-t5k87" (OuterVolumeSpecName: "kube-api-access-t5k87") pod "280f3887-9106-434c-8088-5086ee526ed0" (UID: "280f3887-9106-434c-8088-5086ee526ed0"). InnerVolumeSpecName "kube-api-access-t5k87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:46:58.885058 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.884995    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "280f3887-9106-434c-8088-5086ee526ed0" (UID: "280f3887-9106-434c-8088-5086ee526ed0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:46:58.984736 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.984699    2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280f3887-9106-434c-8088-5086ee526ed0-istiod-ca-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:58.984736 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.984733    2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-workload-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:58.984736 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.984745    2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280f3887-9106-434c-8088-5086ee526ed0-istio-podinfo\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:58.984919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.984758    2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-istio-envoy\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:58.984919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.984770    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5k87\" (UniqueName: \"kubernetes.io/projected/280f3887-9106-434c-8088-5086ee526ed0-kube-api-access-t5k87\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:58.984919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.984781    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280f3887-9106-434c-8088-5086ee526ed0-istio-token\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:58.984919 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:58.984795    2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280f3887-9106-434c-8088-5086ee526ed0-credential-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:46:59.340084 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.340053    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"]
Sep 10 13:46:59.340423 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.340404    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="280f3887-9106-434c-8088-5086ee526ed0" containerName="istio-proxy"
Sep 10 13:46:59.340423 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.340423    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="280f3887-9106-434c-8088-5086ee526ed0" containerName="istio-proxy"
Sep 10 13:46:59.340596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.340434    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="280f3887-9106-434c-8088-5086ee526ed0" containerName="istio-proxy"
Sep 10 13:46:59.340596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.340442    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="280f3887-9106-434c-8088-5086ee526ed0" containerName="istio-proxy"
Sep 10 13:46:59.340596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.340468    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerName="istio-proxy"
Sep 10 13:46:59.340596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.340473    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerName="istio-proxy"
Sep 10 13:46:59.340596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.340480    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f32a5594-8e5a-4955-8e40-1b552f2e3fff" containerName="revoke-keys"
Sep 10 13:46:59.340596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.340485    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f32a5594-8e5a-4955-8e40-1b552f2e3fff" containerName="revoke-keys"
Sep 10 13:46:59.340596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.340551    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f32a5594-8e5a-4955-8e40-1b552f2e3fff" containerName="revoke-keys"
Sep 10 13:46:59.340596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.340560    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="280f3887-9106-434c-8088-5086ee526ed0" containerName="istio-proxy"
Sep 10 13:46:59.340596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.340567    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8efafcf-75f1-43b9-8e09-7684d5f87ad7" containerName="istio-proxy"
Sep 10 13:46:59.340596 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.340585    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="280f3887-9106-434c-8088-5086ee526ed0" containerName="istio-proxy"
Sep 10 13:46:59.343891 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.343871    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.346491 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.346470    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-41abaf-openshift-default-dockercfg-rtntv\""
Sep 10 13:46:59.374481 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.374459    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"]
Sep 10 13:46:59.489756 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.489723    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-credential-socket\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.490212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.489785    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-workload-socket\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.490212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.489859    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w7hr\" (UniqueName: \"kubernetes.io/projected/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-kube-api-access-9w7hr\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.490212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.489928    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-data\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.490212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.489961    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-token\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.490212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.490026    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istiod-ca-cert\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.490212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.490052    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-envoy\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.490212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.490107    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-workload-certs\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.490212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.490134    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-podinfo\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.591489 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.591414    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-workload-socket\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.591489 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.591454    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w7hr\" (UniqueName: \"kubernetes.io/projected/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-kube-api-access-9w7hr\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.591489 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.591482    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-data\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.592102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.591500    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-token\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.592102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.591539    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istiod-ca-cert\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.592102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.591558    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-envoy\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.592102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.591654    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-workload-certs\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.592102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.591685    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-podinfo\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.592102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.591711    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-credential-socket\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.592102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.591873    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-workload-socket\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.592102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.591933    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-data\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.592102 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.592098    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-credential-socket\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.592547 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.592139    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-workload-certs\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.592547 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.592422    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istiod-ca-cert\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.593800 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.593782    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-envoy\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.594020 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.594004    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-podinfo\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.601547 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.601517    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-token\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.601690 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.601655    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w7hr\" (UniqueName: \"kubernetes.io/projected/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-kube-api-access-9w7hr\") pod \"e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") " pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.603832 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.603809    2551 generic.go:356] "Generic (PLEG): container finished" podID="280f3887-9106-434c-8088-5086ee526ed0" containerID="5e49fd0973a3574729b761487f066a1fd847b4352d4adf3d46fa7fb6c1ae370b" exitCode=0
Sep 10 13:46:59.603942 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.603853    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"
Sep 10 13:46:59.603942 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.603889    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f" event={"ID":"280f3887-9106-434c-8088-5086ee526ed0","Type":"ContainerDied","Data":"5e49fd0973a3574729b761487f066a1fd847b4352d4adf3d46fa7fb6c1ae370b"}
Sep 10 13:46:59.603942 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.603929    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f" event={"ID":"280f3887-9106-434c-8088-5086ee526ed0","Type":"ContainerDied","Data":"46a4556b8c865f48ea35f747d1b2a6665825bc96a96098d029a66c99d7ffda47"}
Sep 10 13:46:59.604117 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.603950    2551 scope.go:117] "RemoveContainer" containerID="5e49fd0973a3574729b761487f066a1fd847b4352d4adf3d46fa7fb6c1ae370b"
Sep 10 13:46:59.615878 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.615859    2551 scope.go:117] "RemoveContainer" containerID="92d97800d426c43fed1c6b7343f4ed3c1c6fc91e28f4adbee4d748208ec4ea0a"
Sep 10 13:46:59.623711 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.623487    2551 scope.go:117] "RemoveContainer" containerID="5e49fd0973a3574729b761487f066a1fd847b4352d4adf3d46fa7fb6c1ae370b"
Sep 10 13:46:59.623821 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:46:59.623801    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e49fd0973a3574729b761487f066a1fd847b4352d4adf3d46fa7fb6c1ae370b\": container with ID starting with 5e49fd0973a3574729b761487f066a1fd847b4352d4adf3d46fa7fb6c1ae370b not found: ID does not exist" containerID="5e49fd0973a3574729b761487f066a1fd847b4352d4adf3d46fa7fb6c1ae370b"
Sep 10 13:46:59.623871 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.623827    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e49fd0973a3574729b761487f066a1fd847b4352d4adf3d46fa7fb6c1ae370b"} err="failed to get container status \"5e49fd0973a3574729b761487f066a1fd847b4352d4adf3d46fa7fb6c1ae370b\": rpc error: code = NotFound desc = could not find container \"5e49fd0973a3574729b761487f066a1fd847b4352d4adf3d46fa7fb6c1ae370b\": container with ID starting with 5e49fd0973a3574729b761487f066a1fd847b4352d4adf3d46fa7fb6c1ae370b not found: ID does not exist"
Sep 10 13:46:59.623871 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.623844    2551 scope.go:117] "RemoveContainer" containerID="92d97800d426c43fed1c6b7343f4ed3c1c6fc91e28f4adbee4d748208ec4ea0a"
Sep 10 13:46:59.624085 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:46:59.624062    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92d97800d426c43fed1c6b7343f4ed3c1c6fc91e28f4adbee4d748208ec4ea0a\": container with ID starting with 92d97800d426c43fed1c6b7343f4ed3c1c6fc91e28f4adbee4d748208ec4ea0a not found: ID does not exist" containerID="92d97800d426c43fed1c6b7343f4ed3c1c6fc91e28f4adbee4d748208ec4ea0a"
Sep 10 13:46:59.624134 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.624092    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92d97800d426c43fed1c6b7343f4ed3c1c6fc91e28f4adbee4d748208ec4ea0a"} err="failed to get container status \"92d97800d426c43fed1c6b7343f4ed3c1c6fc91e28f4adbee4d748208ec4ea0a\": rpc error: code = NotFound desc = could not find container \"92d97800d426c43fed1c6b7343f4ed3c1c6fc91e28f4adbee4d748208ec4ea0a\": container with ID starting with 92d97800d426c43fed1c6b7343f4ed3c1c6fc91e28f4adbee4d748208ec4ea0a not found: ID does not exist"
Sep 10 13:46:59.629846 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.629802    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"]
Sep 10 13:46:59.632521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.632504    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5d75ac-openshift-default-7c5c985c8d-nlt6f"]
Sep 10 13:46:59.654629 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.654609    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:46:59.723908 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.723878    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="280f3887-9106-434c-8088-5086ee526ed0" path="/var/lib/kubelet/pods/280f3887-9106-434c-8088-5086ee526ed0/volumes"
Sep 10 13:46:59.942140 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.942114    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"]
Sep 10 13:46:59.943936 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:46:59.943909    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecff4124_6fb1_4c1c_87f2_16244e4a92c4.slice/crio-02e745116570b269cdd76877de72b69723d4534a92d83ca8de9851b752d7ea48 WatchSource:0}: Error finding container 02e745116570b269cdd76877de72b69723d4534a92d83ca8de9851b752d7ea48: Status 404 returned error can't find the container with id 02e745116570b269cdd76877de72b69723d4534a92d83ca8de9851b752d7ea48
Sep 10 13:46:59.946470 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.946437    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:46:59.946531 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.946503    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:46:59.946585 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:46:59.946530    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:47:00.611704 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:00.611664    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b" event={"ID":"ecff4124-6fb1-4c1c-87f2-16244e4a92c4","Type":"ContainerStarted","Data":"b1d82174f446b2b9f0728b5889c23f5b4064ef82403c3b07c3aa5745dab3c9a7"}
Sep 10 13:47:00.612148 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:00.611721    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b" event={"ID":"ecff4124-6fb1-4c1c-87f2-16244e4a92c4","Type":"ContainerStarted","Data":"02e745116570b269cdd76877de72b69723d4534a92d83ca8de9851b752d7ea48"}
Sep 10 13:47:00.637127 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:00.637067    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b" podStartSLOduration=1.63704807 podStartE2EDuration="1.63704807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:46:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:47:00.636472686 +0000 UTC m=+1731.455511099" watchObservedRunningTime="2026-09-10 13:47:00.63704807 +0000 UTC m=+1731.456086473"
Sep 10 13:47:00.655701 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:00.655680    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:47:00.660772 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:00.660744    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:47:01.617092 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:01.617062    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:47:01.618012 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:01.617995    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:47:04.272706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:04.272677    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:47:05.163057 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:05.163014    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb"]
Sep 10 13:47:05.167180 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:05.167060    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ba174e-jqrfb"]
Sep 10 13:47:05.709773 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:05.709742    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f32a5594-8e5a-4955-8e40-1b552f2e3fff" path="/var/lib/kubelet/pods/f32a5594-8e5a-4955-8e40-1b552f2e3fff/volumes"
Sep 10 13:47:12.545451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.545407    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4" podUID="5db10590-49ef-4528-90b9-c55a9d28affc" containerName="main" containerID="cri-o://84edfc2496922fa280db7cfb8ea85582992b2dd87c163873594e0a46e3d62a82" gracePeriod=45
Sep 10 13:47:12.675107 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.675080    2551 generic.go:356] "Generic (PLEG): container finished" podID="5db10590-49ef-4528-90b9-c55a9d28affc" containerID="84edfc2496922fa280db7cfb8ea85582992b2dd87c163873594e0a46e3d62a82" exitCode=0
Sep 10 13:47:12.675226 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.675148    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4" event={"ID":"5db10590-49ef-4528-90b9-c55a9d28affc","Type":"ContainerDied","Data":"84edfc2496922fa280db7cfb8ea85582992b2dd87c163873594e0a46e3d62a82"}
Sep 10 13:47:12.785743 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.785717    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:47:12.819747 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.819684    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnjnx\" (UniqueName: \"kubernetes.io/projected/5db10590-49ef-4528-90b9-c55a9d28affc-kube-api-access-hnjnx\") pod \"5db10590-49ef-4528-90b9-c55a9d28affc\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") "
Sep 10 13:47:12.819883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.819755    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-tmp-dir\") pod \"5db10590-49ef-4528-90b9-c55a9d28affc\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") "
Sep 10 13:47:12.819883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.819788    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-home\") pod \"5db10590-49ef-4528-90b9-c55a9d28affc\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") "
Sep 10 13:47:12.819883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.819814    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5db10590-49ef-4528-90b9-c55a9d28affc-tls-certs\") pod \"5db10590-49ef-4528-90b9-c55a9d28affc\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") "
Sep 10 13:47:12.819883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.819837    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-dshm\") pod \"5db10590-49ef-4528-90b9-c55a9d28affc\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") "
Sep 10 13:47:12.820100 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.819888    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-model-cache\") pod \"5db10590-49ef-4528-90b9-c55a9d28affc\" (UID: \"5db10590-49ef-4528-90b9-c55a9d28affc\") "
Sep 10 13:47:12.820100 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.820059    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5db10590-49ef-4528-90b9-c55a9d28affc" (UID: "5db10590-49ef-4528-90b9-c55a9d28affc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:47:12.820275 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.820239    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-model-cache" (OuterVolumeSpecName: "model-cache") pod "5db10590-49ef-4528-90b9-c55a9d28affc" (UID: "5db10590-49ef-4528-90b9-c55a9d28affc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:47:12.820275 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.820256    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-home" (OuterVolumeSpecName: "home") pod "5db10590-49ef-4528-90b9-c55a9d28affc" (UID: "5db10590-49ef-4528-90b9-c55a9d28affc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:47:12.820465 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.820307    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-tmp-dir\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:47:12.821911 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.821886    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5db10590-49ef-4528-90b9-c55a9d28affc-kube-api-access-hnjnx" (OuterVolumeSpecName: "kube-api-access-hnjnx") pod "5db10590-49ef-4528-90b9-c55a9d28affc" (UID: "5db10590-49ef-4528-90b9-c55a9d28affc"). InnerVolumeSpecName "kube-api-access-hnjnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:47:12.822000 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.821957    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-dshm" (OuterVolumeSpecName: "dshm") pod "5db10590-49ef-4528-90b9-c55a9d28affc" (UID: "5db10590-49ef-4528-90b9-c55a9d28affc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:47:12.822000 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.821960    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5db10590-49ef-4528-90b9-c55a9d28affc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5db10590-49ef-4528-90b9-c55a9d28affc" (UID: "5db10590-49ef-4528-90b9-c55a9d28affc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:47:12.921784 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.921751    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-home\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:47:12.921784 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.921785    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5db10590-49ef-4528-90b9-c55a9d28affc-tls-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:47:12.921950 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.921799    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-dshm\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:47:12.921950 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.921812    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5db10590-49ef-4528-90b9-c55a9d28affc-model-cache\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:47:12.921950 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:12.921823    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnjnx\" (UniqueName: \"kubernetes.io/projected/5db10590-49ef-4528-90b9-c55a9d28affc-kube-api-access-hnjnx\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:47:13.604510 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.604465    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96" podUID="460358b1-9ba3-48e3-8e8b-92a2b0c1d954" containerName="main" containerID="cri-o://de65da63cf03516455341ee9df1503cbd96272ad36a37ef5ae0b5872c84a3424" gracePeriod=45
Sep 10 13:47:13.681996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.681969    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"
Sep 10 13:47:13.681996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.681981    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4" event={"ID":"5db10590-49ef-4528-90b9-c55a9d28affc","Type":"ContainerDied","Data":"0d6c3f4cafd630c03bcb77a757778e75064062aef871dd190c5114da7682edc2"}
Sep 10 13:47:13.682213 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.682033    2551 scope.go:117] "RemoveContainer" containerID="84edfc2496922fa280db7cfb8ea85582992b2dd87c163873594e0a46e3d62a82"
Sep 10 13:47:13.711094 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.711067    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"]
Sep 10 13:47:13.711220 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.711109    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-767dce/rate-test-model-767dce-kserve-f586ffb8c-lm6v4"]
Sep 10 13:47:13.895698 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.895675    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:47:13.929862 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.929835    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-dshm\") pod \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") "
Sep 10 13:47:13.930011 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.929880    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-tmp-dir\") pod \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") "
Sep 10 13:47:13.930011 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.929920    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9wvg\" (UniqueName: \"kubernetes.io/projected/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-kube-api-access-q9wvg\") pod \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") "
Sep 10 13:47:13.930011 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.929958    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-model-cache\") pod \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") "
Sep 10 13:47:13.930011 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.929991    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-home\") pod \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") "
Sep 10 13:47:13.930231 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.930062    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-tls-certs\") pod \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\" (UID: \"460358b1-9ba3-48e3-8e8b-92a2b0c1d954\") "
Sep 10 13:47:13.930279 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.930218    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "460358b1-9ba3-48e3-8e8b-92a2b0c1d954" (UID: "460358b1-9ba3-48e3-8e8b-92a2b0c1d954"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:47:13.930279 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.930234    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-model-cache" (OuterVolumeSpecName: "model-cache") pod "460358b1-9ba3-48e3-8e8b-92a2b0c1d954" (UID: "460358b1-9ba3-48e3-8e8b-92a2b0c1d954"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:47:13.930279 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.930255    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-home" (OuterVolumeSpecName: "home") pod "460358b1-9ba3-48e3-8e8b-92a2b0c1d954" (UID: "460358b1-9ba3-48e3-8e8b-92a2b0c1d954"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:47:13.930457 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.930436    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-tmp-dir\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:47:13.930492 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.930461    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-model-cache\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:47:13.930492 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.930474    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-home\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:47:13.932181 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.932153    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "460358b1-9ba3-48e3-8e8b-92a2b0c1d954" (UID: "460358b1-9ba3-48e3-8e8b-92a2b0c1d954"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:47:13.932296 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.932211    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-kube-api-access-q9wvg" (OuterVolumeSpecName: "kube-api-access-q9wvg") pod "460358b1-9ba3-48e3-8e8b-92a2b0c1d954" (UID: "460358b1-9ba3-48e3-8e8b-92a2b0c1d954"). InnerVolumeSpecName "kube-api-access-q9wvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:47:13.932296 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:13.932260    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-dshm" (OuterVolumeSpecName: "dshm") pod "460358b1-9ba3-48e3-8e8b-92a2b0c1d954" (UID: "460358b1-9ba3-48e3-8e8b-92a2b0c1d954"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:47:14.031411 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:14.031375    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-tls-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:47:14.031411 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:14.031407    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-dshm\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:47:14.031411 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:14.031415    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9wvg\" (UniqueName: \"kubernetes.io/projected/460358b1-9ba3-48e3-8e8b-92a2b0c1d954-kube-api-access-q9wvg\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:47:14.690014 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:14.689978    2551 generic.go:356] "Generic (PLEG): container finished" podID="460358b1-9ba3-48e3-8e8b-92a2b0c1d954" containerID="de65da63cf03516455341ee9df1503cbd96272ad36a37ef5ae0b5872c84a3424" exitCode=0
Sep 10 13:47:14.690460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:14.690019    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"
Sep 10 13:47:14.690460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:14.690070    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96" event={"ID":"460358b1-9ba3-48e3-8e8b-92a2b0c1d954","Type":"ContainerDied","Data":"de65da63cf03516455341ee9df1503cbd96272ad36a37ef5ae0b5872c84a3424"}
Sep 10 13:47:14.690460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:14.690115    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96" event={"ID":"460358b1-9ba3-48e3-8e8b-92a2b0c1d954","Type":"ContainerDied","Data":"21f6b2265966b1171a7cf345c1845f1caef8236e52b12667a4ea207190bb35ac"}
Sep 10 13:47:14.690460 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:14.690134    2551 scope.go:117] "RemoveContainer" containerID="de65da63cf03516455341ee9df1503cbd96272ad36a37ef5ae0b5872c84a3424"
Sep 10 13:47:14.699060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:14.699046    2551 scope.go:117] "RemoveContainer" containerID="de65da63cf03516455341ee9df1503cbd96272ad36a37ef5ae0b5872c84a3424"
Sep 10 13:47:14.699319 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:47:14.699302    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de65da63cf03516455341ee9df1503cbd96272ad36a37ef5ae0b5872c84a3424\": container with ID starting with de65da63cf03516455341ee9df1503cbd96272ad36a37ef5ae0b5872c84a3424 not found: ID does not exist" containerID="de65da63cf03516455341ee9df1503cbd96272ad36a37ef5ae0b5872c84a3424"
Sep 10 13:47:14.699358 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:14.699326    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de65da63cf03516455341ee9df1503cbd96272ad36a37ef5ae0b5872c84a3424"} err="failed to get container status \"de65da63cf03516455341ee9df1503cbd96272ad36a37ef5ae0b5872c84a3424\": rpc error: code = NotFound desc = could not find container \"de65da63cf03516455341ee9df1503cbd96272ad36a37ef5ae0b5872c84a3424\": container with ID starting with de65da63cf03516455341ee9df1503cbd96272ad36a37ef5ae0b5872c84a3424 not found: ID does not exist"
Sep 10 13:47:14.713146 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:14.713123    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"]
Sep 10 13:47:14.714885 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:14.714866    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-89be92/rate-test-model-89be92-kserve-76db94d467-qcj96"]
Sep 10 13:47:15.709165 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:15.709133    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="460358b1-9ba3-48e3-8e8b-92a2b0c1d954" path="/var/lib/kubelet/pods/460358b1-9ba3-48e3-8e8b-92a2b0c1d954/volumes"
Sep 10 13:47:15.709538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:15.709511    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5db10590-49ef-4528-90b9-c55a9d28affc" path="/var/lib/kubelet/pods/5db10590-49ef-4528-90b9-c55a9d28affc/volumes"
Sep 10 13:47:16.181192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:16.181162    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:47:22.737315 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:22.737277    2551 generic.go:356] "Generic (PLEG): container finished" podID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" containerID="b1d82174f446b2b9f0728b5889c23f5b4064ef82403c3b07c3aa5745dab3c9a7" exitCode=0
Sep 10 13:47:22.737315 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:22.737306    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b" event={"ID":"ecff4124-6fb1-4c1c-87f2-16244e4a92c4","Type":"ContainerDied","Data":"b1d82174f446b2b9f0728b5889c23f5b4064ef82403c3b07c3aa5745dab3c9a7"}
Sep 10 13:47:22.737785 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:22.737673    2551 scope.go:117] "RemoveContainer" containerID="b1d82174f446b2b9f0728b5889c23f5b4064ef82403c3b07c3aa5745dab3c9a7"
Sep 10 13:47:22.738895 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:22.738864    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:47:22.738982 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:22.738954    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:47:22.739023 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:22.738999    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:47:23.744410 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:23.744378    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b" event={"ID":"ecff4124-6fb1-4c1c-87f2-16244e4a92c4","Type":"ContainerStarted","Data":"0e1f71c4826bcc797e6a45eb184331034c0ecf426b275d3e07f486e1b9ee23a6"}
Sep 10 13:47:24.655296 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:24.655262    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:47:24.655660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:24.655635    2551 patch_prober.go:28] interesting pod/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 10 13:47:24.655827 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:24.655800    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b" podUID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 10 13:47:25.655754 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:25.655716    2551 patch_prober.go:28] interesting pod/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 10 13:47:25.656337 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:25.656304    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b" podUID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 10 13:47:26.656012 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:26.655971    2551 patch_prober.go:28] interesting pod/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 10 13:47:26.656483 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:26.656046    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b" podUID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 10 13:47:27.655487 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:27.655448    2551 patch_prober.go:28] interesting pod/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 10 13:47:27.655817 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:27.655783    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b" podUID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 10 13:47:28.560106 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.560070    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"]
Sep 10 13:47:28.561188 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.561163    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5db10590-49ef-4528-90b9-c55a9d28affc" containerName="main"
Sep 10 13:47:28.561336 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.561323    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5db10590-49ef-4528-90b9-c55a9d28affc" containerName="main"
Sep 10 13:47:28.561443 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.561430    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="460358b1-9ba3-48e3-8e8b-92a2b0c1d954" containerName="main"
Sep 10 13:47:28.561534 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.561523    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="460358b1-9ba3-48e3-8e8b-92a2b0c1d954" containerName="main"
Sep 10 13:47:28.561776 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.561760    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5db10590-49ef-4528-90b9-c55a9d28affc" containerName="main"
Sep 10 13:47:28.561888 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.561877    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="460358b1-9ba3-48e3-8e8b-92a2b0c1d954" containerName="main"
Sep 10 13:47:28.570539 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.570464    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.575652 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.575622    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-645758-openshift-default-dockercfg-24j46\""
Sep 10 13:47:28.577963 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.577939    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"]
Sep 10 13:47:28.657954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.657919    2551 patch_prober.go:28] interesting pod/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 10 13:47:28.658236 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.658208    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b" podUID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 10 13:47:28.686875 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.686839    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-envoy\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.687049 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.686881    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-workload-certs\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.687049 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.686960    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-credential-socket\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.687049 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.686994    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bf2da32-d45f-458e-8c79-994a27e2739e-istiod-ca-cert\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.687049 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.687024    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-podinfo\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.687218 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.687092    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-workload-socket\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.687218 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.687129    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-data\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.687218 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.687148    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f5b8\" (UniqueName: \"kubernetes.io/projected/2bf2da32-d45f-458e-8c79-994a27e2739e-kube-api-access-8f5b8\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.687344 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.687252    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-token\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.788390 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.788353    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-podinfo\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.788594 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.788425    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-workload-socket\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.788594 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.788469    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-data\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.788594 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.788500    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f5b8\" (UniqueName: \"kubernetes.io/projected/2bf2da32-d45f-458e-8c79-994a27e2739e-kube-api-access-8f5b8\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.788594 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.788551    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-token\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.788828 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.788637    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-envoy\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.788828 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.788677    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-workload-certs\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.788828 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.788736    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-credential-socket\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.788828 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.788768    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bf2da32-d45f-458e-8c79-994a27e2739e-istiod-ca-cert\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.789034 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.788913    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-data\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.789034 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.788926    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-workload-socket\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.789129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.789111    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-workload-certs\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.789185 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.789150    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-credential-socket\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.789545 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.789525    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bf2da32-d45f-458e-8c79-994a27e2739e-istiod-ca-cert\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.790953 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.790933    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-podinfo\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.791189 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.791167    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-envoy\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.801525 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.801507    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-token\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.804742 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.804720    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f5b8\" (UniqueName: \"kubernetes.io/projected/2bf2da32-d45f-458e-8c79-994a27e2739e-kube-api-access-8f5b8\") pod \"e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") " pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:28.883613 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:28.883526    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:29.171392 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:29.171360    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"]
Sep 10 13:47:29.173107 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:47:29.173082    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bf2da32_d45f_458e_8c79_994a27e2739e.slice/crio-c42d868bf32cb9093d5d8459df10649aad97d3868b132523793e4a60e2ae5c88 WatchSource:0}: Error finding container c42d868bf32cb9093d5d8459df10649aad97d3868b132523793e4a60e2ae5c88: Status 404 returned error can't find the container with id c42d868bf32cb9093d5d8459df10649aad97d3868b132523793e4a60e2ae5c88
Sep 10 13:47:29.175851 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:29.175811    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:47:29.175949 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:29.175897    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:47:29.175949 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:29.175943    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:47:29.659013 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:29.658986    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:47:29.659438 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:29.659273    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:47:29.659977 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:29.659958    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:47:29.786197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:29.786156    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" event={"ID":"2bf2da32-d45f-458e-8c79-994a27e2739e","Type":"ContainerStarted","Data":"1ce7e95e1bfece732fba9379258a84584cd17f5443e448028168615511cd1bf5"}
Sep 10 13:47:29.786383 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:29.786214    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" event={"ID":"2bf2da32-d45f-458e-8c79-994a27e2739e","Type":"ContainerStarted","Data":"c42d868bf32cb9093d5d8459df10649aad97d3868b132523793e4a60e2ae5c88"}
Sep 10 13:47:29.813544 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:29.813502    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" podStartSLOduration=1.8134870699999999 podStartE2EDuration="1.81348707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:47:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:47:29.810720109 +0000 UTC m=+1760.629758511" watchObservedRunningTime="2026-09-10 13:47:29.81348707 +0000 UTC m=+1760.632525461"
Sep 10 13:47:29.884449 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:29.884426    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:30.889435 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:30.889404    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:31.797559 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:31.797530    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:31.798512 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:31.798489    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:39.480451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:39.480418    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:47:47.838721 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:47.838679    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 13:47:47.839145 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:47.838743    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 13:47:49.903070 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:49.903035    2551 generic.go:356] "Generic (PLEG): container finished" podID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerID="1ce7e95e1bfece732fba9379258a84584cd17f5443e448028168615511cd1bf5" exitCode=0
Sep 10 13:47:49.903614 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:49.903078    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" event={"ID":"2bf2da32-d45f-458e-8c79-994a27e2739e","Type":"ContainerDied","Data":"1ce7e95e1bfece732fba9379258a84584cd17f5443e448028168615511cd1bf5"}
Sep 10 13:47:49.903614 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:49.903521    2551 scope.go:117] "RemoveContainer" containerID="1ce7e95e1bfece732fba9379258a84584cd17f5443e448028168615511cd1bf5"
Sep 10 13:47:49.904625 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:49.904569    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:47:49.904731 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:49.904663    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:47:49.904731 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:49.904694    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:47:50.909734 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:50.909698    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" event={"ID":"2bf2da32-d45f-458e-8c79-994a27e2739e","Type":"ContainerStarted","Data":"726c8a5edb229d82f3107693794dfa5532f8d651fc3bfdf58bdfc36248c70a50"}
Sep 10 13:47:51.884774 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:51.884742    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:51.884960 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:51.884937    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 10 13:47:51.885008 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:51.884986    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 10 13:47:52.884423 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:52.884389    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 10 13:47:52.884828 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:52.884450    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 10 13:47:53.884499 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:53.884462    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 10 13:47:53.884913 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:53.884526    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 10 13:47:54.884723 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:54.884683    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 10 13:47:54.885107 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:54.884760    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 10 13:47:55.884362 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:55.884328    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 10 13:47:55.884542 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:55.884396    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 10 13:47:56.887869 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:56.887841    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:56.888289 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:56.888176    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:47:56.889479 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:47:56.889460    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:48:03.778773 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:03.778692    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"]
Sep 10 13:48:03.782334 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:03.782312    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:03.785418 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:03.785394    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-41abaf\"/\"kube-root-ca.crt\""
Sep 10 13:48:03.786531 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:03.786514    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-41abaf\"/\"default-dockercfg-fb6ft\""
Sep 10 13:48:03.786632 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:03.786547    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-41abaf\"/\"ipp-route-41abaf-kserve-self-signed-certs\""
Sep 10 13:48:03.786632 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:03.786557    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-41abaf\"/\"openshift-service-ca.crt\""
Sep 10 13:48:03.795092 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:03.795069    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"]
Sep 10 13:48:03.920858 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:03.920821    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-home\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:03.921005 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:03.920883    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-dshm\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:03.921005 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:03.920984    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ec55842-90c5-4f46-b0a1-82eccb562c2d-tls-certs\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:03.921137 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:03.921020    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2sjq\" (UniqueName: \"kubernetes.io/projected/6ec55842-90c5-4f46-b0a1-82eccb562c2d-kube-api-access-f2sjq\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:03.921137 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:03.921060    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-model-cache\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:03.921250 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:03.921139    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-tmp-dir\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:04.022381 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.022349    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-home\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:04.022555 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.022395    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-dshm\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:04.022555 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.022427    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ec55842-90c5-4f46-b0a1-82eccb562c2d-tls-certs\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:04.022555 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.022442    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2sjq\" (UniqueName: \"kubernetes.io/projected/6ec55842-90c5-4f46-b0a1-82eccb562c2d-kube-api-access-f2sjq\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:04.022555 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.022465    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-model-cache\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:04.022555 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.022511    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-tmp-dir\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:04.022858 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.022826    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-home\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:04.022899 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.022827    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-model-cache\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:04.022935 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.022913    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-tmp-dir\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:04.024663 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.024644    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-dshm\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:04.024947 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.024930    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ec55842-90c5-4f46-b0a1-82eccb562c2d-tls-certs\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:04.033893 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.033838    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2sjq\" (UniqueName: \"kubernetes.io/projected/6ec55842-90c5-4f46-b0a1-82eccb562c2d-kube-api-access-f2sjq\") pod \"ipp-route-41abaf-kserve-778b5bd7ff-g6fzq\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") " pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:04.092040 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.092004    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:04.363484 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.363460    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"]
Sep 10 13:48:04.364996 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:48:04.364971    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ec55842_90c5_4f46_b0a1_82eccb562c2d.slice/crio-a01dfa9ae1fb94a9584236011f1d98080a846125b88ed7d03a7dac0e0fe1051d WatchSource:0}: Error finding container a01dfa9ae1fb94a9584236011f1d98080a846125b88ed7d03a7dac0e0fe1051d: Status 404 returned error can't find the container with id a01dfa9ae1fb94a9584236011f1d98080a846125b88ed7d03a7dac0e0fe1051d
Sep 10 13:48:04.987203 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.987165    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq" event={"ID":"6ec55842-90c5-4f46-b0a1-82eccb562c2d","Type":"ContainerStarted","Data":"0829dfa6656a5e095a5503de498fd5964817997d6b69ed685e9578120fc24f01"}
Sep 10 13:48:04.987203 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:04.987208    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq" event={"ID":"6ec55842-90c5-4f46-b0a1-82eccb562c2d","Type":"ContainerStarted","Data":"a01dfa9ae1fb94a9584236011f1d98080a846125b88ed7d03a7dac0e0fe1051d"}
Sep 10 13:48:05.007546 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:05.007499    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq" podStartSLOduration=2.007487073 podStartE2EDuration="2.007487073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:48:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:48:05.005131054 +0000 UTC m=+1795.824169455" watchObservedRunningTime="2026-09-10 13:48:05.007487073 +0000 UTC m=+1795.826525473"
Sep 10 13:48:08.473972 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:08.473937    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:48:09.754323 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:09.754296    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:48:09.761008 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:09.760986    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:48:14.092229 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:14.092195    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:14.092229 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:14.092235    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:14.104774 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:14.104750    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:15.047509 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:15.047477    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:48:21.360545 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:21.360507    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:48:41.286295 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:41.286256    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:48:57.373964 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:57.373929    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd"]
Sep 10 13:48:57.377521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:57.377504    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd"
Sep 10 13:48:57.380660 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:57.380640    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:48:57.380767 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:57.380638    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:48:57.385109 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:57.385083    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd"]
Sep 10 13:48:57.498970 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:57.498932    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75a832d2-8c1e-411c-8a27-9f824792e537-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd\" (UID: \"75a832d2-8c1e-411c-8a27-9f824792e537\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd"
Sep 10 13:48:57.600491 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:57.600451    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75a832d2-8c1e-411c-8a27-9f824792e537-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd\" (UID: \"75a832d2-8c1e-411c-8a27-9f824792e537\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd"
Sep 10 13:48:57.601089 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:57.601068    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75a832d2-8c1e-411c-8a27-9f824792e537-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd\" (UID: \"75a832d2-8c1e-411c-8a27-9f824792e537\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd"
Sep 10 13:48:57.689735 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:57.689659    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd"
Sep 10 13:48:57.958762 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:57.958735    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd"]
Sep 10 13:48:57.960313 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:48:57.960284    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a832d2_8c1e_411c_8a27_9f824792e537.slice/crio-804af0ea1f2249526a78230a65f0767e3ac2919c07125f07899bff4b0e597275 WatchSource:0}: Error finding container 804af0ea1f2249526a78230a65f0767e3ac2919c07125f07899bff4b0e597275: Status 404 returned error can't find the container with id 804af0ea1f2249526a78230a65f0767e3ac2919c07125f07899bff4b0e597275
Sep 10 13:48:58.264329 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:58.264301    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd" event={"ID":"75a832d2-8c1e-411c-8a27-9f824792e537","Type":"ContainerStarted","Data":"65f6dd939fc2f51e203a227ccba6f9545ee93d7655fd307de13ce0616baff4d8"}
Sep 10 13:48:58.264431 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:58.264351    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd" event={"ID":"75a832d2-8c1e-411c-8a27-9f824792e537","Type":"ContainerStarted","Data":"804af0ea1f2249526a78230a65f0767e3ac2919c07125f07899bff4b0e597275"}
Sep 10 13:48:58.282314 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:58.282272    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd" podStartSLOduration=1.282259662 podStartE2EDuration="1.282259662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:48:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:48:58.28096189 +0000 UTC m=+1849.100000289" watchObservedRunningTime="2026-09-10 13:48:58.282259662 +0000 UTC m=+1849.101298064"
Sep 10 13:48:59.271534 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:59.271494    2551 generic.go:356] "Generic (PLEG): container finished" podID="75a832d2-8c1e-411c-8a27-9f824792e537" containerID="65f6dd939fc2f51e203a227ccba6f9545ee93d7655fd307de13ce0616baff4d8" exitCode=0
Sep 10 13:48:59.271910 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:48:59.271597    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd" event={"ID":"75a832d2-8c1e-411c-8a27-9f824792e537","Type":"ContainerDied","Data":"65f6dd939fc2f51e203a227ccba6f9545ee93d7655fd307de13ce0616baff4d8"}
Sep 10 13:49:00.400139 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:00.400113    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd"
Sep 10 13:49:00.528284 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:00.528210    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75a832d2-8c1e-411c-8a27-9f824792e537-maas-api-service-ca\") pod \"75a832d2-8c1e-411c-8a27-9f824792e537\" (UID: \"75a832d2-8c1e-411c-8a27-9f824792e537\") "
Sep 10 13:49:00.528568 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:00.528542    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75a832d2-8c1e-411c-8a27-9f824792e537-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "75a832d2-8c1e-411c-8a27-9f824792e537" (UID: "75a832d2-8c1e-411c-8a27-9f824792e537"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:49:00.629940 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:00.629911    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75a832d2-8c1e-411c-8a27-9f824792e537-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:49:01.283249 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:01.283223    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd"
Sep 10 13:49:01.283249 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:01.283227    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd" event={"ID":"75a832d2-8c1e-411c-8a27-9f824792e537","Type":"ContainerDied","Data":"804af0ea1f2249526a78230a65f0767e3ac2919c07125f07899bff4b0e597275"}
Sep 10 13:49:01.283456 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:01.283280    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="804af0ea1f2249526a78230a65f0767e3ac2919c07125f07899bff4b0e597275"
Sep 10 13:49:03.454145 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:03.454108    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd"]
Sep 10 13:49:03.458340 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:03.458316    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08e484-7crnd"]
Sep 10 13:49:03.713230 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:03.713155    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75a832d2-8c1e-411c-8a27-9f824792e537" path="/var/lib/kubelet/pods/75a832d2-8c1e-411c-8a27-9f824792e537/volumes"
Sep 10 13:49:07.880533 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:07.880502    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:49:09.982746 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:09.982718    2551 scope.go:117] "RemoveContainer" containerID="fa37a3fe1eb70dbbd6db975fa09252098d9c02edf87a63a4a8e81e15364101a1"
Sep 10 13:49:14.640535 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:14.640499    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj"]
Sep 10 13:49:14.641263 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:14.640871    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75a832d2-8c1e-411c-8a27-9f824792e537" containerName="revoke-keys"
Sep 10 13:49:14.641263 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:14.640882    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="75a832d2-8c1e-411c-8a27-9f824792e537" containerName="revoke-keys"
Sep 10 13:49:14.641263 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:14.640955    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="75a832d2-8c1e-411c-8a27-9f824792e537" containerName="revoke-keys"
Sep 10 13:49:14.643758 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:14.643742    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj"
Sep 10 13:49:14.646367 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:14.646339    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:49:14.646475 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:14.646449    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:49:14.654292 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:14.654263    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj"]
Sep 10 13:49:14.770318 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:14.770282    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42e42aac-599a-4413-8f99-27d48f1a32aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj\" (UID: \"42e42aac-599a-4413-8f99-27d48f1a32aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj"
Sep 10 13:49:14.871015 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:14.870981    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42e42aac-599a-4413-8f99-27d48f1a32aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj\" (UID: \"42e42aac-599a-4413-8f99-27d48f1a32aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj"
Sep 10 13:49:14.871642 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:14.871618    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42e42aac-599a-4413-8f99-27d48f1a32aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj\" (UID: \"42e42aac-599a-4413-8f99-27d48f1a32aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj"
Sep 10 13:49:14.955130 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:14.955023    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj"
Sep 10 13:49:15.239455 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:15.239406    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj"]
Sep 10 13:49:15.355858 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:15.355812    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj" event={"ID":"42e42aac-599a-4413-8f99-27d48f1a32aa","Type":"ContainerStarted","Data":"e3ba83a63e59565d1e73f6f500e0281cc227dcdcb5bcaf7f24e808423dbf0be6"}
Sep 10 13:49:16.362746 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:16.362706    2551 generic.go:356] "Generic (PLEG): container finished" podID="42e42aac-599a-4413-8f99-27d48f1a32aa" containerID="3fac9e1e0afd9cb2d19828ce99026a8fbfba7d2870faa39ea9f33fd36d8c9535" exitCode=0
Sep 10 13:49:16.363188 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:16.362793    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj" event={"ID":"42e42aac-599a-4413-8f99-27d48f1a32aa","Type":"ContainerDied","Data":"3fac9e1e0afd9cb2d19828ce99026a8fbfba7d2870faa39ea9f33fd36d8c9535"}
Sep 10 13:49:17.502272 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:17.502249    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj"
Sep 10 13:49:17.595848 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:17.595816    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42e42aac-599a-4413-8f99-27d48f1a32aa-maas-api-service-ca\") pod \"42e42aac-599a-4413-8f99-27d48f1a32aa\" (UID: \"42e42aac-599a-4413-8f99-27d48f1a32aa\") "
Sep 10 13:49:17.596197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:17.596167    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42e42aac-599a-4413-8f99-27d48f1a32aa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "42e42aac-599a-4413-8f99-27d48f1a32aa" (UID: "42e42aac-599a-4413-8f99-27d48f1a32aa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:49:17.697376 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:17.697290    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42e42aac-599a-4413-8f99-27d48f1a32aa-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:49:18.374382 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:18.374351    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj"
Sep 10 13:49:18.374382 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:18.374362    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj" event={"ID":"42e42aac-599a-4413-8f99-27d48f1a32aa","Type":"ContainerDied","Data":"e3ba83a63e59565d1e73f6f500e0281cc227dcdcb5bcaf7f24e808423dbf0be6"}
Sep 10 13:49:18.374602 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:18.374404    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3ba83a63e59565d1e73f6f500e0281cc227dcdcb5bcaf7f24e808423dbf0be6"
Sep 10 13:49:28.000239 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:28.000197    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:49:30.558400 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:30.558356    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj"]
Sep 10 13:49:30.561238 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:30.561208    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41abaf-v9kjj"]
Sep 10 13:49:31.710280 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:31.710241    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42e42aac-599a-4413-8f99-27d48f1a32aa" path="/var/lib/kubelet/pods/42e42aac-599a-4413-8f99-27d48f1a32aa/volumes"
Sep 10 13:49:35.924672 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:35.924621    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"]
Sep 10 13:49:40.779480 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:40.779437    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:49:50.945451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:50.945406    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq" podUID="6ec55842-90c5-4f46-b0a1-82eccb562c2d" containerName="main" containerID="cri-o://0829dfa6656a5e095a5503de498fd5964817997d6b69ed685e9578120fc24f01" gracePeriod=45
Sep 10 13:49:51.192503 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.192475    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:49:51.226646 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.226617    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-model-cache\") pod \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") "
Sep 10 13:49:51.226807 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.226690    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-home\") pod \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") "
Sep 10 13:49:51.226807 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.226740    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-tmp-dir\") pod \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") "
Sep 10 13:49:51.226807 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.226792    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-dshm\") pod \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") "
Sep 10 13:49:51.226950 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.226822    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ec55842-90c5-4f46-b0a1-82eccb562c2d-tls-certs\") pod \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") "
Sep 10 13:49:51.226950 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.226890    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2sjq\" (UniqueName: \"kubernetes.io/projected/6ec55842-90c5-4f46-b0a1-82eccb562c2d-kube-api-access-f2sjq\") pod \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\" (UID: \"6ec55842-90c5-4f46-b0a1-82eccb562c2d\") "
Sep 10 13:49:51.227057 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.226966    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-model-cache" (OuterVolumeSpecName: "model-cache") pod "6ec55842-90c5-4f46-b0a1-82eccb562c2d" (UID: "6ec55842-90c5-4f46-b0a1-82eccb562c2d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:51.227057 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.227026    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-home" (OuterVolumeSpecName: "home") pod "6ec55842-90c5-4f46-b0a1-82eccb562c2d" (UID: "6ec55842-90c5-4f46-b0a1-82eccb562c2d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:51.227161 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.227081    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6ec55842-90c5-4f46-b0a1-82eccb562c2d" (UID: "6ec55842-90c5-4f46-b0a1-82eccb562c2d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:51.227324 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.227298    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-model-cache\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:49:51.227324 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.227326    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-home\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:49:51.227494 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.227341    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-tmp-dir\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:49:51.229224 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.229196    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ec55842-90c5-4f46-b0a1-82eccb562c2d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6ec55842-90c5-4f46-b0a1-82eccb562c2d" (UID: "6ec55842-90c5-4f46-b0a1-82eccb562c2d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:49:51.229385 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.229365    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-dshm" (OuterVolumeSpecName: "dshm") pod "6ec55842-90c5-4f46-b0a1-82eccb562c2d" (UID: "6ec55842-90c5-4f46-b0a1-82eccb562c2d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:49:51.229598 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.229550    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ec55842-90c5-4f46-b0a1-82eccb562c2d-kube-api-access-f2sjq" (OuterVolumeSpecName: "kube-api-access-f2sjq") pod "6ec55842-90c5-4f46-b0a1-82eccb562c2d" (UID: "6ec55842-90c5-4f46-b0a1-82eccb562c2d"). InnerVolumeSpecName "kube-api-access-f2sjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:49:51.328786 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.328749    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ec55842-90c5-4f46-b0a1-82eccb562c2d-tls-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:49:51.328786 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.328782    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2sjq\" (UniqueName: \"kubernetes.io/projected/6ec55842-90c5-4f46-b0a1-82eccb562c2d-kube-api-access-f2sjq\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:49:51.328786 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.328793    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ec55842-90c5-4f46-b0a1-82eccb562c2d-dshm\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:49:51.552770 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.552676    2551 generic.go:356] "Generic (PLEG): container finished" podID="6ec55842-90c5-4f46-b0a1-82eccb562c2d" containerID="0829dfa6656a5e095a5503de498fd5964817997d6b69ed685e9578120fc24f01" exitCode=0
Sep 10 13:49:51.552770 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.552716    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"
Sep 10 13:49:51.552975 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.552765    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq" event={"ID":"6ec55842-90c5-4f46-b0a1-82eccb562c2d","Type":"ContainerDied","Data":"0829dfa6656a5e095a5503de498fd5964817997d6b69ed685e9578120fc24f01"}
Sep 10 13:49:51.552975 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.552807    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq" event={"ID":"6ec55842-90c5-4f46-b0a1-82eccb562c2d","Type":"ContainerDied","Data":"a01dfa9ae1fb94a9584236011f1d98080a846125b88ed7d03a7dac0e0fe1051d"}
Sep 10 13:49:51.552975 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.552827    2551 scope.go:117] "RemoveContainer" containerID="0829dfa6656a5e095a5503de498fd5964817997d6b69ed685e9578120fc24f01"
Sep 10 13:49:51.563120 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.563084    2551 scope.go:117] "RemoveContainer" containerID="0829dfa6656a5e095a5503de498fd5964817997d6b69ed685e9578120fc24f01"
Sep 10 13:49:51.563472 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:49:51.563452    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0829dfa6656a5e095a5503de498fd5964817997d6b69ed685e9578120fc24f01\": container with ID starting with 0829dfa6656a5e095a5503de498fd5964817997d6b69ed685e9578120fc24f01 not found: ID does not exist" containerID="0829dfa6656a5e095a5503de498fd5964817997d6b69ed685e9578120fc24f01"
Sep 10 13:49:51.563526 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.563483    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0829dfa6656a5e095a5503de498fd5964817997d6b69ed685e9578120fc24f01"} err="failed to get container status \"0829dfa6656a5e095a5503de498fd5964817997d6b69ed685e9578120fc24f01\": rpc error: code = NotFound desc = could not find container \"0829dfa6656a5e095a5503de498fd5964817997d6b69ed685e9578120fc24f01\": container with ID starting with 0829dfa6656a5e095a5503de498fd5964817997d6b69ed685e9578120fc24f01 not found: ID does not exist"
Sep 10 13:49:51.576487 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.576456    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"]
Sep 10 13:49:51.580400 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.580374    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-41abaf/ipp-route-41abaf-kserve-778b5bd7ff-g6fzq"]
Sep 10 13:49:51.713081 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:49:51.713036    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ec55842-90c5-4f46-b0a1-82eccb562c2d" path="/var/lib/kubelet/pods/6ec55842-90c5-4f46-b0a1-82eccb562c2d/volumes"
Sep 10 13:50:19.509429 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:19.509396    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"]
Sep 10 13:50:19.509966 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:19.509733    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b" podUID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" containerName="istio-proxy" containerID="cri-o://0e1f71c4826bcc797e6a45eb184331034c0ecf426b275d3e07f486e1b9ee23a6" gracePeriod=30
Sep 10 13:50:19.865148 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:19.865069    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl"]
Sep 10 13:50:19.865450 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:19.865437    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ec55842-90c5-4f46-b0a1-82eccb562c2d" containerName="main"
Sep 10 13:50:19.865450 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:19.865451    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ec55842-90c5-4f46-b0a1-82eccb562c2d" containerName="main"
Sep 10 13:50:19.865538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:19.865467    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42e42aac-599a-4413-8f99-27d48f1a32aa" containerName="revoke-keys"
Sep 10 13:50:19.865538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:19.865473    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e42aac-599a-4413-8f99-27d48f1a32aa" containerName="revoke-keys"
Sep 10 13:50:19.865538 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:19.865533    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ec55842-90c5-4f46-b0a1-82eccb562c2d" containerName="main"
Sep 10 13:50:19.865657 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:19.865543    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="42e42aac-599a-4413-8f99-27d48f1a32aa" containerName="revoke-keys"
Sep 10 13:50:19.868550 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:19.868530    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl"
Sep 10 13:50:19.871140 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:19.871118    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:50:19.871258 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:19.871121    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:50:19.876380 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:19.876342    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl"]
Sep 10 13:50:19.992693 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:19.992656    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b44a1c47-0d76-4a58-8c72-462faa0fe85c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl\" (UID: \"b44a1c47-0d76-4a58-8c72-462faa0fe85c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl"
Sep 10 13:50:20.093284 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:20.093247    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b44a1c47-0d76-4a58-8c72-462faa0fe85c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl\" (UID: \"b44a1c47-0d76-4a58-8c72-462faa0fe85c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl"
Sep 10 13:50:20.093854 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:20.093836    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b44a1c47-0d76-4a58-8c72-462faa0fe85c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl\" (UID: \"b44a1c47-0d76-4a58-8c72-462faa0fe85c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl"
Sep 10 13:50:20.179641 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:20.179562    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl"
Sep 10 13:50:20.450540 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:20.450517    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl"]
Sep 10 13:50:20.452522 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:50:20.452493    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb44a1c47_0d76_4a58_8c72_462faa0fe85c.slice/crio-053de813ea4cc892ec456578fa43b65555032bce7d5319412fa7f8a438ed975c WatchSource:0}: Error finding container 053de813ea4cc892ec456578fa43b65555032bce7d5319412fa7f8a438ed975c: Status 404 returned error can't find the container with id 053de813ea4cc892ec456578fa43b65555032bce7d5319412fa7f8a438ed975c
Sep 10 13:50:20.704735 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:20.704675    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl" event={"ID":"b44a1c47-0d76-4a58-8c72-462faa0fe85c","Type":"ContainerStarted","Data":"053de813ea4cc892ec456578fa43b65555032bce7d5319412fa7f8a438ed975c"}
Sep 10 13:50:21.713049 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:21.713015    2551 generic.go:356] "Generic (PLEG): container finished" podID="b44a1c47-0d76-4a58-8c72-462faa0fe85c" containerID="4c44663bef1a20463a3b16680a9b6ea42839fe47133a042a0a90f5db38ba8850" exitCode=0
Sep 10 13:50:21.713434 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:21.713094    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl" event={"ID":"b44a1c47-0d76-4a58-8c72-462faa0fe85c","Type":"ContainerDied","Data":"4c44663bef1a20463a3b16680a9b6ea42839fe47133a042a0a90f5db38ba8850"}
Sep 10 13:50:22.845841 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:22.845815    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl"
Sep 10 13:50:22.919083 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:22.919052    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b44a1c47-0d76-4a58-8c72-462faa0fe85c-maas-api-service-ca\") pod \"b44a1c47-0d76-4a58-8c72-462faa0fe85c\" (UID: \"b44a1c47-0d76-4a58-8c72-462faa0fe85c\") "
Sep 10 13:50:22.919365 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:22.919343    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b44a1c47-0d76-4a58-8c72-462faa0fe85c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b44a1c47-0d76-4a58-8c72-462faa0fe85c" (UID: "b44a1c47-0d76-4a58-8c72-462faa0fe85c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:50:23.019734 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:23.019702    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b44a1c47-0d76-4a58-8c72-462faa0fe85c-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:23.723761 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:23.723735    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl"
Sep 10 13:50:23.723883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:23.723761    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl" event={"ID":"b44a1c47-0d76-4a58-8c72-462faa0fe85c","Type":"ContainerDied","Data":"053de813ea4cc892ec456578fa43b65555032bce7d5319412fa7f8a438ed975c"}
Sep 10 13:50:23.723883 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:23.723802    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="053de813ea4cc892ec456578fa43b65555032bce7d5319412fa7f8a438ed975c"
Sep 10 13:50:24.731457 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.731425    2551 generic.go:356] "Generic (PLEG): container finished" podID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" containerID="0e1f71c4826bcc797e6a45eb184331034c0ecf426b275d3e07f486e1b9ee23a6" exitCode=0
Sep 10 13:50:24.731854 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.731498    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b" event={"ID":"ecff4124-6fb1-4c1c-87f2-16244e4a92c4","Type":"ContainerDied","Data":"0e1f71c4826bcc797e6a45eb184331034c0ecf426b275d3e07f486e1b9ee23a6"}
Sep 10 13:50:24.731854 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.731613    2551 scope.go:117] "RemoveContainer" containerID="b1d82174f446b2b9f0728b5889c23f5b4064ef82403c3b07c3aa5745dab3c9a7"
Sep 10 13:50:24.764080 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.764061    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:50:24.838869 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.838793    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-workload-socket\") pod \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") "
Sep 10 13:50:24.838869 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.838854    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istiod-ca-cert\") pod \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") "
Sep 10 13:50:24.839082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.838890    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-token\") pod \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") "
Sep 10 13:50:24.839082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.838913    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-data\") pod \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") "
Sep 10 13:50:24.839082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.839013    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ecff4124-6fb1-4c1c-87f2-16244e4a92c4" (UID: "ecff4124-6fb1-4c1c-87f2-16244e4a92c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:50:24.839082 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.839021    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w7hr\" (UniqueName: \"kubernetes.io/projected/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-kube-api-access-9w7hr\") pod \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") "
Sep 10 13:50:24.839307 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.839099    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-podinfo\") pod \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") "
Sep 10 13:50:24.839307 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.839149    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-workload-certs\") pod \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") "
Sep 10 13:50:24.839307 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.839187    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-envoy\") pod \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") "
Sep 10 13:50:24.839307 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.839243    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-credential-socket\") pod \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\" (UID: \"ecff4124-6fb1-4c1c-87f2-16244e4a92c4\") "
Sep 10 13:50:24.839307 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.839251    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-data" (OuterVolumeSpecName: "istio-data") pod "ecff4124-6fb1-4c1c-87f2-16244e4a92c4" (UID: "ecff4124-6fb1-4c1c-87f2-16244e4a92c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:50:24.839307 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.839283    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ecff4124-6fb1-4c1c-87f2-16244e4a92c4" (UID: "ecff4124-6fb1-4c1c-87f2-16244e4a92c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:50:24.839669 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.839486    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ecff4124-6fb1-4c1c-87f2-16244e4a92c4" (UID: "ecff4124-6fb1-4c1c-87f2-16244e4a92c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:50:24.839669 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.839624    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ecff4124-6fb1-4c1c-87f2-16244e4a92c4" (UID: "ecff4124-6fb1-4c1c-87f2-16244e4a92c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:50:24.839777 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.839682    2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-workload-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:24.839777 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.839702    2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-workload-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:24.839777 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.839716    2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istiod-ca-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:24.839777 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.839730    2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-data\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:24.841197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.841168    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "ecff4124-6fb1-4c1c-87f2-16244e4a92c4" (UID: "ecff4124-6fb1-4c1c-87f2-16244e4a92c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:50:24.841322 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.841279    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-kube-api-access-9w7hr" (OuterVolumeSpecName: "kube-api-access-9w7hr") pod "ecff4124-6fb1-4c1c-87f2-16244e4a92c4" (UID: "ecff4124-6fb1-4c1c-87f2-16244e4a92c4"). InnerVolumeSpecName "kube-api-access-9w7hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:50:24.841379 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.841318    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ecff4124-6fb1-4c1c-87f2-16244e4a92c4" (UID: "ecff4124-6fb1-4c1c-87f2-16244e4a92c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 13:50:24.841469 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.841451    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ecff4124-6fb1-4c1c-87f2-16244e4a92c4" (UID: "ecff4124-6fb1-4c1c-87f2-16244e4a92c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:50:24.940164 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.940143    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-token\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:24.940164 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.940165    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w7hr\" (UniqueName: \"kubernetes.io/projected/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-kube-api-access-9w7hr\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:24.940164 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.940175    2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-podinfo\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:24.940346 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.940183    2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-istio-envoy\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:24.940346 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:24.940192    2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecff4124-6fb1-4c1c-87f2-16244e4a92c4-credential-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:25.738514 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:25.738490    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"
Sep 10 13:50:25.738514 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:25.738498    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b" event={"ID":"ecff4124-6fb1-4c1c-87f2-16244e4a92c4","Type":"ContainerDied","Data":"02e745116570b269cdd76877de72b69723d4534a92d83ca8de9851b752d7ea48"}
Sep 10 13:50:25.738938 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:25.738555    2551 scope.go:117] "RemoveContainer" containerID="0e1f71c4826bcc797e6a45eb184331034c0ecf426b275d3e07f486e1b9ee23a6"
Sep 10 13:50:25.760841 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:25.760817    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"]
Sep 10 13:50:25.764996 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:25.764976    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-41abaf-openshift-default-67fc88d568-5kg6b"]
Sep 10 13:50:27.709706 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:27.709663    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" path="/var/lib/kubelet/pods/ecff4124-6fb1-4c1c-87f2-16244e4a92c4/volumes"
Sep 10 13:50:30.744244 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:30.744207    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl"]
Sep 10 13:50:30.747294 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:30.747269    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-645758-tbtzl"]
Sep 10 13:50:31.709694 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:31.709654    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b44a1c47-0d76-4a58-8c72-462faa0fe85c" path="/var/lib/kubelet/pods/b44a1c47-0d76-4a58-8c72-462faa0fe85c/volumes"
Sep 10 13:50:33.763842 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:33.763811    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:50:37.583824 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:37.583794    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"]
Sep 10 13:50:37.584189 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:37.584017    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerName="istio-proxy" containerID="cri-o://726c8a5edb229d82f3107693794dfa5532f8d651fc3bfdf58bdfc36248c70a50" gracePeriod=30
Sep 10 13:50:37.948865 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:37.948783    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl"]
Sep 10 13:50:37.949141 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:37.949129    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" containerName="istio-proxy"
Sep 10 13:50:37.949192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:37.949143    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" containerName="istio-proxy"
Sep 10 13:50:37.949192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:37.949171    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b44a1c47-0d76-4a58-8c72-462faa0fe85c" containerName="revoke-keys"
Sep 10 13:50:37.949192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:37.949177    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b44a1c47-0d76-4a58-8c72-462faa0fe85c" containerName="revoke-keys"
Sep 10 13:50:37.949287 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:37.949234    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" containerName="istio-proxy"
Sep 10 13:50:37.949287 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:37.949243    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="b44a1c47-0d76-4a58-8c72-462faa0fe85c" containerName="revoke-keys"
Sep 10 13:50:37.949287 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:37.949253    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" containerName="istio-proxy"
Sep 10 13:50:37.953940 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:37.953921    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl"
Sep 10 13:50:37.956622 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:37.956597    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:50:37.956877 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:37.956862    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:50:37.962129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:37.961850    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl"]
Sep 10 13:50:38.056311 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:38.056276    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ff0307c-60ca-47d6-ac6a-67b561cc505f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl\" (UID: \"8ff0307c-60ca-47d6-ac6a-67b561cc505f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl"
Sep 10 13:50:38.157722 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:38.157689    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ff0307c-60ca-47d6-ac6a-67b561cc505f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl\" (UID: \"8ff0307c-60ca-47d6-ac6a-67b561cc505f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl"
Sep 10 13:50:38.158299 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:38.158279    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ff0307c-60ca-47d6-ac6a-67b561cc505f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl\" (UID: \"8ff0307c-60ca-47d6-ac6a-67b561cc505f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl"
Sep 10 13:50:38.266813 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:38.266781    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl"
Sep 10 13:50:38.558469 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:38.558444    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl"]
Sep 10 13:50:38.561250 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:50:38.561224    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ff0307c_60ca_47d6_ac6a_67b561cc505f.slice/crio-c21d1cc40cfc50c9949bd491cf2f2b659bb0ae104805a6b91530a9b27734d4fb WatchSource:0}: Error finding container c21d1cc40cfc50c9949bd491cf2f2b659bb0ae104805a6b91530a9b27734d4fb: Status 404 returned error can't find the container with id c21d1cc40cfc50c9949bd491cf2f2b659bb0ae104805a6b91530a9b27734d4fb
Sep 10 13:50:38.806052 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:38.806021    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl" event={"ID":"8ff0307c-60ca-47d6-ac6a-67b561cc505f","Type":"ContainerStarted","Data":"c21d1cc40cfc50c9949bd491cf2f2b659bb0ae104805a6b91530a9b27734d4fb"}
Sep 10 13:50:39.812899 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:39.812868    2551 generic.go:356] "Generic (PLEG): container finished" podID="8ff0307c-60ca-47d6-ac6a-67b561cc505f" containerID="958b0fbe6f9740e90e4cf8b2f0796c6cf062463c2489335a48ffa855356fabf1" exitCode=0
Sep 10 13:50:39.813273 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:39.812960    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl" event={"ID":"8ff0307c-60ca-47d6-ac6a-67b561cc505f","Type":"ContainerDied","Data":"958b0fbe6f9740e90e4cf8b2f0796c6cf062463c2489335a48ffa855356fabf1"}
Sep 10 13:50:40.943317 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:40.943296    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl"
Sep 10 13:50:40.980665 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:40.980642    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ff0307c-60ca-47d6-ac6a-67b561cc505f-maas-api-service-ca\") pod \"8ff0307c-60ca-47d6-ac6a-67b561cc505f\" (UID: \"8ff0307c-60ca-47d6-ac6a-67b561cc505f\") "
Sep 10 13:50:40.980995 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:40.980974    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ff0307c-60ca-47d6-ac6a-67b561cc505f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8ff0307c-60ca-47d6-ac6a-67b561cc505f" (UID: "8ff0307c-60ca-47d6-ac6a-67b561cc505f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:50:41.082370 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:41.082302    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ff0307c-60ca-47d6-ac6a-67b561cc505f-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:41.824135 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:41.824105    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl"
Sep 10 13:50:41.824318 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:41.824112    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl" event={"ID":"8ff0307c-60ca-47d6-ac6a-67b561cc505f","Type":"ContainerDied","Data":"c21d1cc40cfc50c9949bd491cf2f2b659bb0ae104805a6b91530a9b27734d4fb"}
Sep 10 13:50:41.824318 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:41.824223    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c21d1cc40cfc50c9949bd491cf2f2b659bb0ae104805a6b91530a9b27734d4fb"
Sep 10 13:50:41.890269 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:41.890237    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 13:50:41.890398 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:41.890293    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 13:50:42.831887 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.831856    2551 generic.go:356] "Generic (PLEG): container finished" podID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerID="726c8a5edb229d82f3107693794dfa5532f8d651fc3bfdf58bdfc36248c70a50" exitCode=0
Sep 10 13:50:42.832192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.831928    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" event={"ID":"2bf2da32-d45f-458e-8c79-994a27e2739e","Type":"ContainerDied","Data":"726c8a5edb229d82f3107693794dfa5532f8d651fc3bfdf58bdfc36248c70a50"}
Sep 10 13:50:42.832192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.832013    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m" event={"ID":"2bf2da32-d45f-458e-8c79-994a27e2739e","Type":"ContainerDied","Data":"c42d868bf32cb9093d5d8459df10649aad97d3868b132523793e4a60e2ae5c88"}
Sep 10 13:50:42.832192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.832032    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c42d868bf32cb9093d5d8459df10649aad97d3868b132523793e4a60e2ae5c88"
Sep 10 13:50:42.832192 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.832047    2551 scope.go:117] "RemoveContainer" containerID="1ce7e95e1bfece732fba9379258a84584cd17f5443e448028168615511cd1bf5"
Sep 10 13:50:42.833965 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.833946    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:50:42.895871 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.895807    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-workload-certs\") pod \"2bf2da32-d45f-458e-8c79-994a27e2739e\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") "
Sep 10 13:50:42.895871 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.895844    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-data\") pod \"2bf2da32-d45f-458e-8c79-994a27e2739e\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") "
Sep 10 13:50:42.896063 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.895888    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-token\") pod \"2bf2da32-d45f-458e-8c79-994a27e2739e\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") "
Sep 10 13:50:42.896063 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.895909    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-credential-socket\") pod \"2bf2da32-d45f-458e-8c79-994a27e2739e\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") "
Sep 10 13:50:42.896063 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.895954    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-envoy\") pod \"2bf2da32-d45f-458e-8c79-994a27e2739e\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") "
Sep 10 13:50:42.896063 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.895998    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bf2da32-d45f-458e-8c79-994a27e2739e-istiod-ca-cert\") pod \"2bf2da32-d45f-458e-8c79-994a27e2739e\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") "
Sep 10 13:50:42.896063 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.896020    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8f5b8\" (UniqueName: \"kubernetes.io/projected/2bf2da32-d45f-458e-8c79-994a27e2739e-kube-api-access-8f5b8\") pod \"2bf2da32-d45f-458e-8c79-994a27e2739e\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") "
Sep 10 13:50:42.896063 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.896038    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-workload-socket\") pod \"2bf2da32-d45f-458e-8c79-994a27e2739e\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") "
Sep 10 13:50:42.896063 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.896065    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-podinfo\") pod \"2bf2da32-d45f-458e-8c79-994a27e2739e\" (UID: \"2bf2da32-d45f-458e-8c79-994a27e2739e\") "
Sep 10 13:50:42.896432 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.896122    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2bf2da32-d45f-458e-8c79-994a27e2739e" (UID: "2bf2da32-d45f-458e-8c79-994a27e2739e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:50:42.896432 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.896194    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-data" (OuterVolumeSpecName: "istio-data") pod "2bf2da32-d45f-458e-8c79-994a27e2739e" (UID: "2bf2da32-d45f-458e-8c79-994a27e2739e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:50:42.896432 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.896399    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2bf2da32-d45f-458e-8c79-994a27e2739e" (UID: "2bf2da32-d45f-458e-8c79-994a27e2739e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:50:42.896432 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.896412    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2bf2da32-d45f-458e-8c79-994a27e2739e" (UID: "2bf2da32-d45f-458e-8c79-994a27e2739e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:50:42.896432 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.896424    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bf2da32-d45f-458e-8c79-994a27e2739e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2bf2da32-d45f-458e-8c79-994a27e2739e" (UID: "2bf2da32-d45f-458e-8c79-994a27e2739e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:50:42.896698 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.896438    2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-workload-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:42.896698 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.896458    2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-data\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:42.898236 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.898212    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2bf2da32-d45f-458e-8c79-994a27e2739e" (UID: "2bf2da32-d45f-458e-8c79-994a27e2739e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:50:42.898379 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.898360    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bf2da32-d45f-458e-8c79-994a27e2739e-kube-api-access-8f5b8" (OuterVolumeSpecName: "kube-api-access-8f5b8") pod "2bf2da32-d45f-458e-8c79-994a27e2739e" (UID: "2bf2da32-d45f-458e-8c79-994a27e2739e"). InnerVolumeSpecName "kube-api-access-8f5b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:50:42.898453 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.898412    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-token" (OuterVolumeSpecName: "istio-token") pod "2bf2da32-d45f-458e-8c79-994a27e2739e" (UID: "2bf2da32-d45f-458e-8c79-994a27e2739e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:50:42.898453 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.898412    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2bf2da32-d45f-458e-8c79-994a27e2739e" (UID: "2bf2da32-d45f-458e-8c79-994a27e2739e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 13:50:42.996873 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.996848    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-token\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:42.996873 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.996869    2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-credential-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:42.997016 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.996879    2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-envoy\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:42.997016 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.996887    2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bf2da32-d45f-458e-8c79-994a27e2739e-istiod-ca-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:42.997016 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.996894    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8f5b8\" (UniqueName: \"kubernetes.io/projected/2bf2da32-d45f-458e-8c79-994a27e2739e-kube-api-access-8f5b8\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:42.997016 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.996902    2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bf2da32-d45f-458e-8c79-994a27e2739e-workload-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:42.997016 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:42.996910    2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bf2da32-d45f-458e-8c79-994a27e2739e-istio-podinfo\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:43.838989 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:43.838957    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"
Sep 10 13:50:43.873243 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:43.873211    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"]
Sep 10 13:50:43.877131 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:43.877107    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-645758-openshift-default-85f4f9c986-9595m"]
Sep 10 13:50:45.709209 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:45.709176    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" path="/var/lib/kubelet/pods/2bf2da32-d45f-458e-8c79-994a27e2739e/volumes"
Sep 10 13:50:48.580567 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:48.580532    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-767dce-29817465-hjsh2"]
Sep 10 13:50:48.584746 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:48.584720    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-767dce-29817465-hjsh2"]
Sep 10 13:50:48.793219 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:48.793185    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl"]
Sep 10 13:50:48.796355 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:48.796330    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-767dce-4m9dl"]
Sep 10 13:50:49.709542 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:49.709499    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ff0307c-60ca-47d6-ac6a-67b561cc505f" path="/var/lib/kubelet/pods/8ff0307c-60ca-47d6-ac6a-67b561cc505f/volumes"
Sep 10 13:50:49.710029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:49.709936    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf8bbef1-caf2-4a08-800c-bcad76e29ea2" path="/var/lib/kubelet/pods/bf8bbef1-caf2-4a08-800c-bcad76e29ea2/volumes"
Sep 10 13:50:51.675554 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:51.675522    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:50:56.082543 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.082507    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl"]
Sep 10 13:50:56.082932 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.082860    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" containerName="istio-proxy"
Sep 10 13:50:56.082932 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.082873    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecff4124-6fb1-4c1c-87f2-16244e4a92c4" containerName="istio-proxy"
Sep 10 13:50:56.082932 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.082883    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff0307c-60ca-47d6-ac6a-67b561cc505f" containerName="revoke-keys"
Sep 10 13:50:56.082932 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.082888    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff0307c-60ca-47d6-ac6a-67b561cc505f" containerName="revoke-keys"
Sep 10 13:50:56.082932 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.082901    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerName="istio-proxy"
Sep 10 13:50:56.082932 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.082907    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerName="istio-proxy"
Sep 10 13:50:56.082932 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.082921    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerName="istio-proxy"
Sep 10 13:50:56.082932 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.082928    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerName="istio-proxy"
Sep 10 13:50:56.083182 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.082978    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerName="istio-proxy"
Sep 10 13:50:56.083182 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.082985    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bf2da32-d45f-458e-8c79-994a27e2739e" containerName="istio-proxy"
Sep 10 13:50:56.083182 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.082994    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff0307c-60ca-47d6-ac6a-67b561cc505f" containerName="revoke-keys"
Sep 10 13:50:56.087255 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.087237    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl"
Sep 10 13:50:56.089937 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.089915    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:50:56.090060 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.090028    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\""
Sep 10 13:50:56.093976 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.093948    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl"]
Sep 10 13:50:56.206451 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.206417    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3a3495f-e82b-44a1-8795-345c91bc68f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl\" (UID: \"e3a3495f-e82b-44a1-8795-345c91bc68f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl"
Sep 10 13:50:56.307940 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.307899    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3a3495f-e82b-44a1-8795-345c91bc68f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl\" (UID: \"e3a3495f-e82b-44a1-8795-345c91bc68f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl"
Sep 10 13:50:56.308499 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.308471    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3a3495f-e82b-44a1-8795-345c91bc68f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl\" (UID: \"e3a3495f-e82b-44a1-8795-345c91bc68f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl"
Sep 10 13:50:56.399278 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.399208    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl"
Sep 10 13:50:56.669601 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.669560    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl"]
Sep 10 13:50:56.672263 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:50:56.672238    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3a3495f_e82b_44a1_8795_345c91bc68f8.slice/crio-88fca1dd9c78512ef13828b8ba62ccebffc8fabc39fdff7b14a0cb0bbddd94b8 WatchSource:0}: Error finding container 88fca1dd9c78512ef13828b8ba62ccebffc8fabc39fdff7b14a0cb0bbddd94b8: Status 404 returned error can't find the container with id 88fca1dd9c78512ef13828b8ba62ccebffc8fabc39fdff7b14a0cb0bbddd94b8
Sep 10 13:50:56.907807 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:56.907772    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl" event={"ID":"e3a3495f-e82b-44a1-8795-345c91bc68f8","Type":"ContainerStarted","Data":"88fca1dd9c78512ef13828b8ba62ccebffc8fabc39fdff7b14a0cb0bbddd94b8"}
Sep 10 13:50:57.914232 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:57.914192    2551 generic.go:356] "Generic (PLEG): container finished" podID="e3a3495f-e82b-44a1-8795-345c91bc68f8" containerID="ebe0a224c67367ff83338833ae16cde0a2959f47a6004882a1ae92b319b20196" exitCode=0
Sep 10 13:50:57.914604 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:57.914274    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl" event={"ID":"e3a3495f-e82b-44a1-8795-345c91bc68f8","Type":"ContainerDied","Data":"ebe0a224c67367ff83338833ae16cde0a2959f47a6004882a1ae92b319b20196"}
Sep 10 13:50:59.045196 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:59.045175    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl"
Sep 10 13:50:59.131416 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:59.131387    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3a3495f-e82b-44a1-8795-345c91bc68f8-maas-api-service-ca\") pod \"e3a3495f-e82b-44a1-8795-345c91bc68f8\" (UID: \"e3a3495f-e82b-44a1-8795-345c91bc68f8\") "
Sep 10 13:50:59.131763 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:59.131741    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3a3495f-e82b-44a1-8795-345c91bc68f8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e3a3495f-e82b-44a1-8795-345c91bc68f8" (UID: "e3a3495f-e82b-44a1-8795-345c91bc68f8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:50:59.232891 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:59.232870    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3a3495f-e82b-44a1-8795-345c91bc68f8-maas-api-service-ca\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:50:59.926557 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:59.926529    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl"
Sep 10 13:50:59.926557 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:59.926540    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl" event={"ID":"e3a3495f-e82b-44a1-8795-345c91bc68f8","Type":"ContainerDied","Data":"88fca1dd9c78512ef13828b8ba62ccebffc8fabc39fdff7b14a0cb0bbddd94b8"}
Sep 10 13:50:59.926762 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:50:59.926607    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88fca1dd9c78512ef13828b8ba62ccebffc8fabc39fdff7b14a0cb0bbddd94b8"
Sep 10 13:51:06.582083 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:06.581899    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-89be92-29817465-bbpfh"]
Sep 10 13:51:06.583818 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:06.583798    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-89be92-29817465-bbpfh"]
Sep 10 13:51:06.791521 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:06.791487    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl"]
Sep 10 13:51:06.794270 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:06.794248    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-89be92-lx5vl"]
Sep 10 13:51:07.710201 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:07.710169    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="984544c4-7d1d-444f-8643-9835ca6a4aec" path="/var/lib/kubelet/pods/984544c4-7d1d-444f-8643-9835ca6a4aec/volumes"
Sep 10 13:51:07.710565 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:07.710446    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3a3495f-e82b-44a1-8795-345c91bc68f8" path="/var/lib/kubelet/pods/e3a3495f-e82b-44a1-8795-345c91bc68f8/volumes"
Sep 10 13:51:09.258334 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:09.258301    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:51:10.014327 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:10.014300    2551 scope.go:117] "RemoveContainer" containerID="d27cefb371abcce15bc17ccb1714893b25c3c0d4214fa03b1fac1940d461474a"
Sep 10 13:51:10.022708 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:10.022689    2551 scope.go:117] "RemoveContainer" containerID="34a2da22da6080844b4ddbe755adf69d33bff57bfbacaf352b16c772be6b77c8"
Sep 10 13:51:13.813197 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:13.813164    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"]
Sep 10 13:51:13.813701 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:13.813372    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerName="istio-proxy" containerID="cri-o://a9c53306c0d2f8269251aad70d3cb71bb6287adb09da79f5824f6cde3801c619" gracePeriod=30
Sep 10 13:51:16.171117 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:16.171078    2551 patch_prober.go:28] interesting pod/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 13:51:16.171457 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:16.171139    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 13:51:18.874808 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:18.874777    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:51:19.030380 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.030350    2551 generic.go:356] "Generic (PLEG): container finished" podID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerID="a9c53306c0d2f8269251aad70d3cb71bb6287adb09da79f5824f6cde3801c619" exitCode=0
Sep 10 13:51:19.030525 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.030413    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" event={"ID":"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1","Type":"ContainerDied","Data":"a9c53306c0d2f8269251aad70d3cb71bb6287adb09da79f5824f6cde3801c619"}
Sep 10 13:51:19.030525 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.030461    2551 scope.go:117] "RemoveContainer" containerID="4d9d23f72bbb5813b3f079d03c1fd7e7bc449060180a010fc0390f59317686d7"
Sep 10 13:51:19.061728 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.061709    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:51:19.102048 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.101987    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-envoy\") pod \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") "
Sep 10 13:51:19.102048 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102033    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-workload-certs\") pod \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") "
Sep 10 13:51:19.102212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102096    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v4bj\" (UniqueName: \"kubernetes.io/projected/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-kube-api-access-6v4bj\") pod \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") "
Sep 10 13:51:19.102212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102133    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istiod-ca-cert\") pod \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") "
Sep 10 13:51:19.102212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102159    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-podinfo\") pod \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") "
Sep 10 13:51:19.102212 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102200    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-credential-socket\") pod \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") "
Sep 10 13:51:19.102411 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102232    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-workload-socket\") pod \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") "
Sep 10 13:51:19.102411 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102282    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-token\") pod \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") "
Sep 10 13:51:19.102411 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102345    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-data\") pod \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\" (UID: \"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1\") "
Sep 10 13:51:19.102411 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102374    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" (UID: "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:51:19.102630 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102568    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" (UID: "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:51:19.102719 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102690    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-data" (OuterVolumeSpecName: "istio-data") pod "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" (UID: "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:51:19.102796 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102777    2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-workload-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:51:19.102854 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102804    2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-workload-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:51:19.102854 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102818    2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-data\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:51:19.102959 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.102893    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" (UID: "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:51:19.103064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.103044    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" (UID: "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:51:19.104304 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.104276    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-kube-api-access-6v4bj" (OuterVolumeSpecName: "kube-api-access-6v4bj") pod "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" (UID: "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1"). InnerVolumeSpecName "kube-api-access-6v4bj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:51:19.104447 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.104428    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" (UID: "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 13:51:19.104513 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.104468    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" (UID: "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:51:19.105011 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.104997    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" (UID: "f3507b40-69d2-4c9a-9bbf-18c572d5e7e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:51:19.203632 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.203594    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v4bj\" (UniqueName: \"kubernetes.io/projected/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-kube-api-access-6v4bj\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:51:19.203632 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.203633    2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istiod-ca-cert\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:51:19.203632 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.203645    2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-podinfo\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:51:19.203815 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.203652    2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-credential-socket\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:51:19.203815 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.203662    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-token\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:51:19.203815 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:19.203671    2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1-istio-envoy\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:51:20.037318 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:20.037291    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"
Sep 10 13:51:20.037759 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:20.037336    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg" event={"ID":"f3507b40-69d2-4c9a-9bbf-18c572d5e7e1","Type":"ContainerDied","Data":"2446af26344481cc1ed5c91cfec004bdc621cb9f9d34d3355aea60d4659640a9"}
Sep 10 13:51:20.037759 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:20.037391    2551 scope.go:117] "RemoveContainer" containerID="a9c53306c0d2f8269251aad70d3cb71bb6287adb09da79f5824f6cde3801c619"
Sep 10 13:51:20.058487 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:20.058461    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"]
Sep 10 13:51:20.061182 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:20.061157    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-89be92-openshift-default-75f459b984-p85rg"]
Sep 10 13:51:21.709261 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:51:21.709229    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" path="/var/lib/kubelet/pods/f3507b40-69d2-4c9a-9bbf-18c572d5e7e1/volumes"
Sep 10 13:52:10.051163 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:52:10.051133    2551 scope.go:117] "RemoveContainer" containerID="0fb06c256335107c250b7ecf04e8e52e82e99343620d133605ae1a3d5c2fbc13"
Sep 10 13:52:41.176316 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:52:41.176285    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:53:07.070858 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:53:07.070825    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:53:09.790639 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:53:09.790609    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:53:09.796685 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:53:09.796665    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtb95_5ec389cc-200a-4541-a957-8f419ee31f55/kube-multus/0.log"
Sep 10 13:53:10.074961 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:53:10.074885    2551 scope.go:117] "RemoveContainer" containerID="fe618a21bbd0818af786ec19fcdef627a7a57195a79edc27057fccb3fd3f44f7"
Sep 10 13:53:10.082840 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:53:10.082822    2551 scope.go:117] "RemoveContainer" containerID="46e06c2828a0c8fa06583649718c68a8b65eae0d407ccaaa0be5cb28e2e54a4e"
Sep 10 13:53:10.089877 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:53:10.089859    2551 scope.go:117] "RemoveContainer" containerID="af0c0438fed846fefbfc4719c2c108b3c8ab7d05cd2923982cab6ae4b72a8903"
Sep 10 13:53:14.866368 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:53:14.866327    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:53:21.971018 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:53:21.970981    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:53:28.668949 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:53:28.668911    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:53:41.859963 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:53:41.859928    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:53:49.765619 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:53:49.765566    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:54:10.113482 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:54:10.113449    2551 scope.go:117] "RemoveContainer" containerID="726c8a5edb229d82f3107693794dfa5532f8d651fc3bfdf58bdfc36248c70a50"
Sep 10 13:55:10.131961 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:10.131930    2551 scope.go:117] "RemoveContainer" containerID="65f6dd939fc2f51e203a227ccba6f9545ee93d7655fd307de13ce0616baff4d8"
Sep 10 13:55:15.455962 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:15.455924    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:55:23.560431 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:23.560393    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:55:29.468241 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:29.468208    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:55:47.465695 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:47.465607    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:55:48.170234 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.170199    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"]
Sep 10 13:55:48.170650 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.170629    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerName="istio-proxy"
Sep 10 13:55:48.170650 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.170651    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerName="istio-proxy"
Sep 10 13:55:48.170811 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.170674    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerName="istio-proxy"
Sep 10 13:55:48.170811 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.170682    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerName="istio-proxy"
Sep 10 13:55:48.170811 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.170694    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3a3495f-e82b-44a1-8795-345c91bc68f8" containerName="revoke-keys"
Sep 10 13:55:48.170811 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.170703    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3a3495f-e82b-44a1-8795-345c91bc68f8" containerName="revoke-keys"
Sep 10 13:55:48.171007 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.170819    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerName="istio-proxy"
Sep 10 13:55:48.171007 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.170832    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3507b40-69d2-4c9a-9bbf-18c572d5e7e1" containerName="istio-proxy"
Sep 10 13:55:48.171007 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.170844    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3a3495f-e82b-44a1-8795-345c91bc68f8" containerName="revoke-keys"
Sep 10 13:55:48.173952 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.173929    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.176753 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.176730    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-e9da5e77-kserve-self-signed-certs\""
Sep 10 13:55:48.185645 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.185621    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"]
Sep 10 13:55:48.190357 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.190336    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/029c7933-42a5-4163-9fb6-e02b3480afa8-tls-certs\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.190439 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.190368    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-home\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.190439 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.190390    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b68l9\" (UniqueName: \"kubernetes.io/projected/029c7933-42a5-4163-9fb6-e02b3480afa8-kube-api-access-b68l9\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.190439 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.190416    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-tmp-dir\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.190626 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.190561    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-dshm\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.190671 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.190639    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-model-cache\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.291683 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.291646    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-tmp-dir\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.291861 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.291736    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-dshm\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.291861 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.291774    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-model-cache\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.291861 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.291837    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/029c7933-42a5-4163-9fb6-e02b3480afa8-tls-certs\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.292045 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.291870    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-home\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.292045 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.292000    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b68l9\" (UniqueName: \"kubernetes.io/projected/029c7933-42a5-4163-9fb6-e02b3480afa8-kube-api-access-b68l9\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.292771 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.292349    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-home\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.292771 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.292443    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-tmp-dir\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.292771 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.292504    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-model-cache\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.294811 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.294785    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-dshm\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.295044 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.295023    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/029c7933-42a5-4163-9fb6-e02b3480afa8-tls-certs\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.302366 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.302333    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b68l9\" (UniqueName: \"kubernetes.io/projected/029c7933-42a5-4163-9fb6-e02b3480afa8-kube-api-access-b68l9\") pod \"e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") " pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.483377 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.483328    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:48.796349 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:48.796262    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"]
Sep 10 13:55:48.800859 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:55:48.800831    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod029c7933_42a5_4163_9fb6_e02b3480afa8.slice/crio-64ff44028e6a4f47bd0cf2a2539e9aad9c8fe7789e0d33f4d31691e4e7734480 WatchSource:0}: Error finding container 64ff44028e6a4f47bd0cf2a2539e9aad9c8fe7789e0d33f4d31691e4e7734480: Status 404 returned error can't find the container with id 64ff44028e6a4f47bd0cf2a2539e9aad9c8fe7789e0d33f4d31691e4e7734480
Sep 10 13:55:49.358070 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:49.358024    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr" event={"ID":"029c7933-42a5-4163-9fb6-e02b3480afa8","Type":"ContainerStarted","Data":"d4566e1bf481aed12bf7378696eef2ab5d5f427360f3a3515ec2364d21cb35b7"}
Sep 10 13:55:49.358070 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:49.358074    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr" event={"ID":"029c7933-42a5-4163-9fb6-e02b3480afa8","Type":"ContainerStarted","Data":"64ff44028e6a4f47bd0cf2a2539e9aad9c8fe7789e0d33f4d31691e4e7734480"}
Sep 10 13:55:49.377309 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:49.377263    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr" podStartSLOduration=1.3772495949999999 podStartE2EDuration="1.377249595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:55:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:55:49.375978237 +0000 UTC m=+2260.195016638" watchObservedRunningTime="2026-09-10 13:55:49.377249595 +0000 UTC m=+2260.196287995"
Sep 10 13:55:57.262383 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:57.262349    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:55:58.483692 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:58.483650    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:58.484112 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:58.483732    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:58.496135 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:58.496100    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:55:59.416688 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:55:59.416656    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:56:05.215481 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:05.215447    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"]
Sep 10 13:56:06.355265 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:06.355231    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:56:10.152443 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:10.152411    2551 scope.go:117] "RemoveContainer" containerID="3fac9e1e0afd9cb2d19828ce99026a8fbfba7d2870faa39ea9f33fd36d8c9535"
Sep 10 13:56:14.867873 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:14.867840    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:56:20.234085 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.234012    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr" podUID="029c7933-42a5-4163-9fb6-e02b3480afa8" containerName="main" containerID="cri-o://d4566e1bf481aed12bf7378696eef2ab5d5f427360f3a3515ec2364d21cb35b7" gracePeriod=45
Sep 10 13:56:20.481627 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.481603    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:56:20.504612 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.504529    2551 generic.go:356] "Generic (PLEG): container finished" podID="029c7933-42a5-4163-9fb6-e02b3480afa8" containerID="d4566e1bf481aed12bf7378696eef2ab5d5f427360f3a3515ec2364d21cb35b7" exitCode=0
Sep 10 13:56:20.504612 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.504592    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"
Sep 10 13:56:20.504810 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.504609    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr" event={"ID":"029c7933-42a5-4163-9fb6-e02b3480afa8","Type":"ContainerDied","Data":"d4566e1bf481aed12bf7378696eef2ab5d5f427360f3a3515ec2364d21cb35b7"}
Sep 10 13:56:20.504810 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.504648    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr" event={"ID":"029c7933-42a5-4163-9fb6-e02b3480afa8","Type":"ContainerDied","Data":"64ff44028e6a4f47bd0cf2a2539e9aad9c8fe7789e0d33f4d31691e4e7734480"}
Sep 10 13:56:20.504810 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.504670    2551 scope.go:117] "RemoveContainer" containerID="d4566e1bf481aed12bf7378696eef2ab5d5f427360f3a3515ec2364d21cb35b7"
Sep 10 13:56:20.512822 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.512709    2551 scope.go:117] "RemoveContainer" containerID="d4566e1bf481aed12bf7378696eef2ab5d5f427360f3a3515ec2364d21cb35b7"
Sep 10 13:56:20.512989 ip-10-0-135-117 kubenswrapper[2551]: E0910 13:56:20.512968    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4566e1bf481aed12bf7378696eef2ab5d5f427360f3a3515ec2364d21cb35b7\": container with ID starting with d4566e1bf481aed12bf7378696eef2ab5d5f427360f3a3515ec2364d21cb35b7 not found: ID does not exist" containerID="d4566e1bf481aed12bf7378696eef2ab5d5f427360f3a3515ec2364d21cb35b7"
Sep 10 13:56:20.513068 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.512994    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4566e1bf481aed12bf7378696eef2ab5d5f427360f3a3515ec2364d21cb35b7"} err="failed to get container status \"d4566e1bf481aed12bf7378696eef2ab5d5f427360f3a3515ec2364d21cb35b7\": rpc error: code = NotFound desc = could not find container \"d4566e1bf481aed12bf7378696eef2ab5d5f427360f3a3515ec2364d21cb35b7\": container with ID starting with d4566e1bf481aed12bf7378696eef2ab5d5f427360f3a3515ec2364d21cb35b7 not found: ID does not exist"
Sep 10 13:56:20.589025 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.588990    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b68l9\" (UniqueName: \"kubernetes.io/projected/029c7933-42a5-4163-9fb6-e02b3480afa8-kube-api-access-b68l9\") pod \"029c7933-42a5-4163-9fb6-e02b3480afa8\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") "
Sep 10 13:56:20.589121 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.589091    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-home\") pod \"029c7933-42a5-4163-9fb6-e02b3480afa8\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") "
Sep 10 13:56:20.589174 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.589135    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-model-cache\") pod \"029c7933-42a5-4163-9fb6-e02b3480afa8\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") "
Sep 10 13:56:20.589174 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.589167    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/029c7933-42a5-4163-9fb6-e02b3480afa8-tls-certs\") pod \"029c7933-42a5-4163-9fb6-e02b3480afa8\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") "
Sep 10 13:56:20.589248 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.589200    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-dshm\") pod \"029c7933-42a5-4163-9fb6-e02b3480afa8\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") "
Sep 10 13:56:20.589291 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.589252    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-tmp-dir\") pod \"029c7933-42a5-4163-9fb6-e02b3480afa8\" (UID: \"029c7933-42a5-4163-9fb6-e02b3480afa8\") "
Sep 10 13:56:20.589396 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.589372    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-home" (OuterVolumeSpecName: "home") pod "029c7933-42a5-4163-9fb6-e02b3480afa8" (UID: "029c7933-42a5-4163-9fb6-e02b3480afa8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:56:20.589469 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.589379    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-model-cache" (OuterVolumeSpecName: "model-cache") pod "029c7933-42a5-4163-9fb6-e02b3480afa8" (UID: "029c7933-42a5-4163-9fb6-e02b3480afa8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:56:20.589618 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.589592    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "029c7933-42a5-4163-9fb6-e02b3480afa8" (UID: "029c7933-42a5-4163-9fb6-e02b3480afa8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:56:20.589712 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.589622    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-home\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:56:20.589712 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.589642    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-model-cache\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:56:20.591134 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.591113    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/029c7933-42a5-4163-9fb6-e02b3480afa8-kube-api-access-b68l9" (OuterVolumeSpecName: "kube-api-access-b68l9") pod "029c7933-42a5-4163-9fb6-e02b3480afa8" (UID: "029c7933-42a5-4163-9fb6-e02b3480afa8"). InnerVolumeSpecName "kube-api-access-b68l9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:56:20.591222 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.591202    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-dshm" (OuterVolumeSpecName: "dshm") pod "029c7933-42a5-4163-9fb6-e02b3480afa8" (UID: "029c7933-42a5-4163-9fb6-e02b3480afa8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:56:20.591222 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.591208    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/029c7933-42a5-4163-9fb6-e02b3480afa8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "029c7933-42a5-4163-9fb6-e02b3480afa8" (UID: "029c7933-42a5-4163-9fb6-e02b3480afa8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:56:20.690440 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.690406    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/029c7933-42a5-4163-9fb6-e02b3480afa8-tls-certs\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:56:20.690440 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.690437    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-dshm\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:56:20.690593 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.690461    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029c7933-42a5-4163-9fb6-e02b3480afa8-tmp-dir\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:56:20.690593 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.690470    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b68l9\" (UniqueName: \"kubernetes.io/projected/029c7933-42a5-4163-9fb6-e02b3480afa8-kube-api-access-b68l9\") on node \"ip-10-0-135-117.ec2.internal\" DevicePath \"\""
Sep 10 13:56:20.826532 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.826508    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"]
Sep 10 13:56:20.831129 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:20.831108    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-e9da5e77-kserve-5db5874d6c-bsbhr"]
Sep 10 13:56:21.710332 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:21.710301    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="029c7933-42a5-4163-9fb6-e02b3480afa8" path="/var/lib/kubelet/pods/029c7933-42a5-4163-9fb6-e02b3480afa8/volumes"
Sep 10 13:56:24.169633 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:24.169600    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:56:31.963734 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:31.963697    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:56:38.861380 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:38.861342    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:56:44.878719 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:44.878682    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:56:51.963207 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:51.963173    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5jzq6"]
Sep 10 13:56:54.293357 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:56:54.293313    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8c58dd797-lstql_f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6/authorino/0.log"
Sep 10 13:57:00.378756 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:00.378720    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84d444f55-8mtls_ae416f04-8a04-4e64-9e25-aa8bdacdc13b/copy-manifests/0.log"
Sep 10 13:57:00.397987 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:00.397956    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84d444f55-8mtls_ae416f04-8a04-4e64-9e25-aa8bdacdc13b/manager/0.log"
Sep 10 13:57:00.948021 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:00.947994    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817465-kp5xh_a7c11da9-02cd-4291-8c97-c4e012a7acb7/cleanup/0.log"
Sep 10 13:57:01.831516 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:01.831485    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8c58dd797-lstql_f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6/authorino/0.log"
Sep 10 13:57:02.416946 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:02.416919    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-5jzq6_8db4d60e-dff3-43c6-a241-022afc2147d9/limitador/0.log"
Sep 10 13:57:03.243210 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:03.243182    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cb8b5959f-p7pl4_89ee133c-2fe4-4ee1-8638-41fe4bc24fcd/kube-auth-proxy/0.log"
Sep 10 13:57:03.377820 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:03.377790    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jhkw6_36051b6e-ece1-419c-9a57-3610df345db8/istio-proxy/1.log"
Sep 10 13:57:04.336404 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:04.336373    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-c9mtf_6f633bbf-c464-4379-967e-9e4b68d2c8ba/main/0.log"
Sep 10 13:57:04.568115 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:04.568082    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9cgrm_dd1db447-4f7a-42fb-bb74-d4a6610bc09b/main/0.log"
Sep 10 13:57:04.799638 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:04.799609    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-6fcgp_98f4f9fd-0d9f-42d9-90f9-0148df14a83b/main/0.log"
Sep 10 13:57:10.170761 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:10.170731    2551 scope.go:117] "RemoveContainer" containerID="958b0fbe6f9740e90e4cf8b2f0796c6cf062463c2489335a48ffa855356fabf1"
Sep 10 13:57:10.178557 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:10.178535    2551 scope.go:117] "RemoveContainer" containerID="4c44663bef1a20463a3b16680a9b6ea42839fe47133a042a0a90f5db38ba8850"
Sep 10 13:57:10.185609 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:10.185590    2551 scope.go:117] "RemoveContainer" containerID="ebe0a224c67367ff83338833ae16cde0a2959f47a6004882a1ae92b319b20196"
Sep 10 13:57:10.796229 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:10.796195    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-n2zxf/must-gather-trshf"]
Sep 10 13:57:10.796552 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:10.796540    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="029c7933-42a5-4163-9fb6-e02b3480afa8" containerName="main"
Sep 10 13:57:10.796618 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:10.796554    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="029c7933-42a5-4163-9fb6-e02b3480afa8" containerName="main"
Sep 10 13:57:10.796656 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:10.796625    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="029c7933-42a5-4163-9fb6-e02b3480afa8" containerName="main"
Sep 10 13:57:10.801174 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:10.801157    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n2zxf/must-gather-trshf"
Sep 10 13:57:10.804038 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:10.804012    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n2zxf\"/\"kube-root-ca.crt\""
Sep 10 13:57:10.804188 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:10.804012    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n2zxf\"/\"openshift-service-ca.crt\""
Sep 10 13:57:10.804188 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:10.804061    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-n2zxf\"/\"default-dockercfg-frlhp\""
Sep 10 13:57:10.809619 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:10.809597    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n2zxf/must-gather-trshf"]
Sep 10 13:57:10.919933 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:10.919904    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47ss4\" (UniqueName: \"kubernetes.io/projected/08dacea9-9ae4-4c89-b88d-64fb2f98a3f2-kube-api-access-47ss4\") pod \"must-gather-trshf\" (UID: \"08dacea9-9ae4-4c89-b88d-64fb2f98a3f2\") " pod="openshift-must-gather-n2zxf/must-gather-trshf"
Sep 10 13:57:10.920064 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:10.919942    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/08dacea9-9ae4-4c89-b88d-64fb2f98a3f2-must-gather-output\") pod \"must-gather-trshf\" (UID: \"08dacea9-9ae4-4c89-b88d-64fb2f98a3f2\") " pod="openshift-must-gather-n2zxf/must-gather-trshf"
Sep 10 13:57:11.020729 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:11.020687    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47ss4\" (UniqueName: \"kubernetes.io/projected/08dacea9-9ae4-4c89-b88d-64fb2f98a3f2-kube-api-access-47ss4\") pod \"must-gather-trshf\" (UID: \"08dacea9-9ae4-4c89-b88d-64fb2f98a3f2\") " pod="openshift-must-gather-n2zxf/must-gather-trshf"
Sep 10 13:57:11.020931 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:11.020738    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/08dacea9-9ae4-4c89-b88d-64fb2f98a3f2-must-gather-output\") pod \"must-gather-trshf\" (UID: \"08dacea9-9ae4-4c89-b88d-64fb2f98a3f2\") " pod="openshift-must-gather-n2zxf/must-gather-trshf"
Sep 10 13:57:11.021029 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:11.021011    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/08dacea9-9ae4-4c89-b88d-64fb2f98a3f2-must-gather-output\") pod \"must-gather-trshf\" (UID: \"08dacea9-9ae4-4c89-b88d-64fb2f98a3f2\") " pod="openshift-must-gather-n2zxf/must-gather-trshf"
Sep 10 13:57:11.030501 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:11.030467    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47ss4\" (UniqueName: \"kubernetes.io/projected/08dacea9-9ae4-4c89-b88d-64fb2f98a3f2-kube-api-access-47ss4\") pod \"must-gather-trshf\" (UID: \"08dacea9-9ae4-4c89-b88d-64fb2f98a3f2\") " pod="openshift-must-gather-n2zxf/must-gather-trshf"
Sep 10 13:57:11.111363 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:11.111300    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n2zxf/must-gather-trshf"
Sep 10 13:57:11.390880 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:11.390854    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n2zxf/must-gather-trshf"]
Sep 10 13:57:11.392462 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:57:11.392435    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08dacea9_9ae4_4c89_b88d_64fb2f98a3f2.slice/crio-4452e5b6912c03b6c7ac9043b92785a282f03dce3b47b80c6c5c8839b08d7fdf WatchSource:0}: Error finding container 4452e5b6912c03b6c7ac9043b92785a282f03dce3b47b80c6c5c8839b08d7fdf: Status 404 returned error can't find the container with id 4452e5b6912c03b6c7ac9043b92785a282f03dce3b47b80c6c5c8839b08d7fdf
Sep 10 13:57:11.394341 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:11.394325    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 13:57:11.742457 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:11.742423    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n2zxf/must-gather-trshf" event={"ID":"08dacea9-9ae4-4c89-b88d-64fb2f98a3f2","Type":"ContainerStarted","Data":"4452e5b6912c03b6c7ac9043b92785a282f03dce3b47b80c6c5c8839b08d7fdf"}
Sep 10 13:57:12.754915 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:12.754879    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n2zxf/must-gather-trshf" event={"ID":"08dacea9-9ae4-4c89-b88d-64fb2f98a3f2","Type":"ContainerStarted","Data":"a40069c4912615f9eddb0b7110c64c97c64a97e072649b6f37639fc455638355"}
Sep 10 13:57:12.755314 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:12.754930    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n2zxf/must-gather-trshf" event={"ID":"08dacea9-9ae4-4c89-b88d-64fb2f98a3f2","Type":"ContainerStarted","Data":"1c7e4bf6e29f6f8606836eca379e39cd38f1eb99df266cfff08d5ea0810f64aa"}
Sep 10 13:57:12.772311 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:12.772259    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-n2zxf/must-gather-trshf" podStartSLOduration=2.064570902 podStartE2EDuration="2.772241238s" totalImagesPullingTime="707.670336ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:57:12.771054735 +0000 UTC m=+2343.590093136" watchObservedRunningTime="2026-09-10 13:57:12.772241238 +0000 UTC m=+2343.591279639"
Sep 10 13:57:13.756647 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:13.756608    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fnfwb_68de88a2-c141-4c07-9bfd-9ad2aa25b835/global-pull-secret-syncer/0.log"
Sep 10 13:57:13.937517 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:13.937480    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-t85bm_60d72c3e-783a-4a4c-918c-02070def1e7b/konnectivity-agent/0.log"
Sep 10 13:57:14.024946 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:14.024869    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-117.ec2.internal_4a08d7fc3d09b5f1580b5cd035f70a19/haproxy/0.log"
Sep 10 13:57:18.524156 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:18.524071    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8c58dd797-lstql_f4e2cf2d-42d5-4a9b-82a6-d3dbf13623b6/authorino/0.log"
Sep 10 13:57:18.748962 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:18.748920    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-5jzq6_8db4d60e-dff3-43c6-a241-022afc2147d9/limitador/0.log"
Sep 10 13:57:20.736620 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:20.736589    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-8479b9f7c4-68qk5_f4aed292-e888-4f34-86af-29d1e02d5b90/metrics-server/0.log"
Sep 10 13:57:20.800979 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:20.800954    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-xnknx_c0b64459-3f7a-4e4d-9225-c9fe33220957/monitoring-plugin/0.log"
Sep 10 13:57:20.901167 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:20.900647    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hlkvn_21c4a985-fa27-4fcb-bd5c-660cd4906719/node-exporter/0.log"
Sep 10 13:57:20.944593 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:20.944199    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hlkvn_21c4a985-fa27-4fcb-bd5c-660cd4906719/kube-rbac-proxy/0.log"
Sep 10 13:57:20.991939 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:20.991859    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hlkvn_21c4a985-fa27-4fcb-bd5c-660cd4906719/init-textfile/0.log"
Sep 10 13:57:21.343347 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.343316    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_87d2daa4-8508-4aab-9993-46a044dc0f16/prometheus/0.log"
Sep 10 13:57:21.370338 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.370308    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_87d2daa4-8508-4aab-9993-46a044dc0f16/config-reloader/0.log"
Sep 10 13:57:21.397972 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.397944    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_87d2daa4-8508-4aab-9993-46a044dc0f16/thanos-sidecar/0.log"
Sep 10 13:57:21.423973 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.423929    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_87d2daa4-8508-4aab-9993-46a044dc0f16/kube-rbac-proxy-web/0.log"
Sep 10 13:57:21.449543 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.449513    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_87d2daa4-8508-4aab-9993-46a044dc0f16/kube-rbac-proxy/0.log"
Sep 10 13:57:21.478317 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.478286    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_87d2daa4-8508-4aab-9993-46a044dc0f16/kube-rbac-proxy-thanos/0.log"
Sep 10 13:57:21.503903 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.503872    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_87d2daa4-8508-4aab-9993-46a044dc0f16/init-config-reloader/0.log"
Sep 10 13:57:21.622315 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.622291    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55c5f86bf5-fmgwn_eec91424-2065-41b5-bb95-b977d2940fef/telemeter-client/0.log"
Sep 10 13:57:21.653044 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.653016    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55c5f86bf5-fmgwn_eec91424-2065-41b5-bb95-b977d2940fef/reload/0.log"
Sep 10 13:57:21.677081 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.677046    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55c5f86bf5-fmgwn_eec91424-2065-41b5-bb95-b977d2940fef/kube-rbac-proxy/0.log"
Sep 10 13:57:21.717151 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.717121    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bddc647f4-scbfg_8a6461b6-32a7-4faa-8beb-69c46e3324a9/thanos-query/0.log"
Sep 10 13:57:21.746854 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.746788    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bddc647f4-scbfg_8a6461b6-32a7-4faa-8beb-69c46e3324a9/kube-rbac-proxy-web/0.log"
Sep 10 13:57:21.773951 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.773869    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bddc647f4-scbfg_8a6461b6-32a7-4faa-8beb-69c46e3324a9/kube-rbac-proxy/0.log"
Sep 10 13:57:21.798440 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.798409    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bddc647f4-scbfg_8a6461b6-32a7-4faa-8beb-69c46e3324a9/prom-label-proxy/0.log"
Sep 10 13:57:21.827675 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.827631    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bddc647f4-scbfg_8a6461b6-32a7-4faa-8beb-69c46e3324a9/kube-rbac-proxy-rules/0.log"
Sep 10 13:57:21.854828 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:21.854796    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bddc647f4-scbfg_8a6461b6-32a7-4faa-8beb-69c46e3324a9/kube-rbac-proxy-metrics/0.log"
Sep 10 13:57:23.818418 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.818382    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"]
Sep 10 13:57:23.825871 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.825839    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.835616 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.835266    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"]
Sep 10 13:57:23.866462 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.866417    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht9wg\" (UniqueName: \"kubernetes.io/projected/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-kube-api-access-ht9wg\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.866757 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.866735    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-podres\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.866934 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.866914    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-proc\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.867067 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.867052    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-lib-modules\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.867196 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.867183    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-sys\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.968110 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.968071    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-sys\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.968346 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.968316    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-sys\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.968563 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.968540    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht9wg\" (UniqueName: \"kubernetes.io/projected/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-kube-api-access-ht9wg\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.968769 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.968744    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-podres\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.968895 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.968811    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-proc\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.968895 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.968855    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-lib-modules\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.969043 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.969021    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-lib-modules\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.969115 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.969089    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-proc\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.969115 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.969102    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-podres\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:23.979123 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:23.979098    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht9wg\" (UniqueName: \"kubernetes.io/projected/57b8d3b7-e19f-409d-b08d-ffa7b50980ec-kube-api-access-ht9wg\") pod \"perf-node-gather-daemonset-8rz4k\" (UID: \"57b8d3b7-e19f-409d-b08d-ffa7b50980ec\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:24.145818 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:24.145723    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:24.410970 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:24.410886    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6cbf75fcbc-j6svp_83577162-25b2-427a-8f9e-ae6530121d97/console/0.log"
Sep 10 13:57:24.432004 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:24.431969    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"]
Sep 10 13:57:24.436387 ip-10-0-135-117 kubenswrapper[2551]: W0910 13:57:24.436353    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod57b8d3b7_e19f_409d_b08d_ffa7b50980ec.slice/crio-3d95d646e87b603ec376e0abb1c6ce3afae6509270b09904a5724aa6880947fb WatchSource:0}: Error finding container 3d95d646e87b603ec376e0abb1c6ce3afae6509270b09904a5724aa6880947fb: Status 404 returned error can't find the container with id 3d95d646e87b603ec376e0abb1c6ce3afae6509270b09904a5724aa6880947fb
Sep 10 13:57:24.458643 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:24.458619    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-ks28m_75a92c91-c7a6-4b56-b4b1-7437ce6ccb85/download-server/0.log"
Sep 10 13:57:24.844321 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:24.844282    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k" event={"ID":"57b8d3b7-e19f-409d-b08d-ffa7b50980ec","Type":"ContainerStarted","Data":"7c7bf2697a1d11e35cb40c7b935a1b57aaabbea67f87a962cdeed241b10469a6"}
Sep 10 13:57:24.844863 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:24.844351    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k" event={"ID":"57b8d3b7-e19f-409d-b08d-ffa7b50980ec","Type":"ContainerStarted","Data":"3d95d646e87b603ec376e0abb1c6ce3afae6509270b09904a5724aa6880947fb"}
Sep 10 13:57:24.844863 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:24.844400    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:24.863002 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:24.862951    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k" podStartSLOduration=1.86293362 podStartE2EDuration="1.86293362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:57:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:57:24.862164243 +0000 UTC m=+2355.681202645" watchObservedRunningTime="2026-09-10 13:57:24.86293362 +0000 UTC m=+2355.681972020"
Sep 10 13:57:26.318848 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:26.318819    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9whh7_095d9660-3d0b-48eb-abf9-ddd7b60ec7b9/dns/0.log"
Sep 10 13:57:26.345398 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:26.345370    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9whh7_095d9660-3d0b-48eb-abf9-ddd7b60ec7b9/kube-rbac-proxy/0.log"
Sep 10 13:57:26.445227 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:26.445199    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cz6jf_ae6bfe75-19cb-49df-978c-1f369f73429f/dns-node-resolver/0.log"
Sep 10 13:57:27.102955 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:27.102922    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l9cw9_4a5a18c4-1ee9-4a34-b304-49b3f6f26674/node-ca/0.log"
Sep 10 13:57:28.389834 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:28.389804    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cb8b5959f-p7pl4_89ee133c-2fe4-4ee1-8638-41fe4bc24fcd/kube-auth-proxy/0.log"
Sep 10 13:57:28.421095 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:28.421047    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jhkw6_36051b6e-ece1-419c-9a57-3610df345db8/istio-proxy/0.log"
Sep 10 13:57:28.438627 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:28.438591    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jhkw6_36051b6e-ece1-419c-9a57-3610df345db8/istio-proxy/1.log"
Sep 10 13:57:29.461398 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:29.461355    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r7sq8_7dff7693-9534-488c-b000-5ddd0bb3abad/serve-healthcheck-canary/0.log"
Sep 10 13:57:30.171594 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:30.171551    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zszcl_e5ea87df-d83a-4e39-9719-41b92cdfbd3a/kube-rbac-proxy/0.log"
Sep 10 13:57:30.194342 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:30.194309    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zszcl_e5ea87df-d83a-4e39-9719-41b92cdfbd3a/exporter/0.log"
Sep 10 13:57:30.217814 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:30.217784    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zszcl_e5ea87df-d83a-4e39-9719-41b92cdfbd3a/extractor/0.log"
Sep 10 13:57:30.861738 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:30.861711    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8rz4k"
Sep 10 13:57:33.175709 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:33.175307    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84d444f55-8mtls_ae416f04-8a04-4e64-9e25-aa8bdacdc13b/manager/0.log"
Sep 10 13:57:33.196453 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:33.196422    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84d444f55-8mtls_ae416f04-8a04-4e64-9e25-aa8bdacdc13b/copy-manifests/0.log"
Sep 10 13:57:34.437191 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:34.437128    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cb8b5959f-p7pl4_89ee133c-2fe4-4ee1-8638-41fe4bc24fcd/kube-auth-proxy/0.log"
Sep 10 13:57:34.457799 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:34.457748    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jhkw6_36051b6e-ece1-419c-9a57-3610df345db8/istio-proxy/0.log"
Sep 10 13:57:34.476954 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:34.476924    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jhkw6_36051b6e-ece1-419c-9a57-3610df345db8/istio-proxy/1.log"
Sep 10 13:57:34.533313 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:34.533289    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6746899c78-fgqq8_d18d8f64-cfa1-4e94-b938-b00fd21cc581/manager/0.log"
Sep 10 13:57:34.560367 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:34.560331    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rmgbv_4c977b95-db5d-46c8-8b51-ea95c64e9790/openshift-lws-operator/0.log"
Sep 10 13:57:36.286970 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:36.286930    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cb8b5959f-p7pl4_89ee133c-2fe4-4ee1-8638-41fe4bc24fcd/kube-auth-proxy/0.log"
Sep 10 13:57:36.312280 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:36.312249    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jhkw6_36051b6e-ece1-419c-9a57-3610df345db8/istio-proxy/0.log"
Sep 10 13:57:36.330948 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:36.330918    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jhkw6_36051b6e-ece1-419c-9a57-3610df345db8/istio-proxy/1.log"
Sep 10 13:57:41.454322 ip-10-0-135-117 kubenswrapper[2551]: I0910 13:57:41.454290    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ln5qf_d37fb884-c078-4fd0-baf5-cd0ee01f2a7c/kube-multus-additional-cni-plugins/0.log"